 Andrew Cantino
|
815c6d6b86 |
larger model; fix bootstrap error errors |
10 gadi atpakaļ |
 Akinori MUSHA
|
c21bada226 |
WebsiteAgent: Provide a variable _response_ for interpolation. |
10 gadi atpakaļ |
|
Akinori MUSHA
|
e2f2ff5da8 |
Remove "locals" from Drop classes and introduce a real context class. |
10 gadi atpakaļ |
|
Akinori MUSHA
|
0227496669 |
Clean up code with `mode`. |
10 gadi atpakaļ |
|
Akinori MUSHA
|
432f952b0a |
Refactor WebsiteAgent#check_url. |
10 gadi atpakaļ |
|
Akinori MUSHA
|
863e2b8c70 |
WebsiteAgent should interpolate values from incoming event payload. |
10 gadi atpakaļ |
|
Akinori MUSHA
|
ba7dbceb7a |
Omit option details from event_description of WebsiteAgent. |
10 gadi atpakaļ |
|
Akinori MUSHA
|
cb71c84842 |
Remove trailing commas from JSON examples. |
10 gadi atpakaļ |
|
Akinori MUSHA
|
0f1b30bd9c |
Move require's for faraday to the right place. |
10 gadi atpakaļ |
|
Akinori MUSHA
|
39046da035 |
Explain the basics of Ruby's regexp and leave a pointer to ruby-doc.org. |
10 gadi atpakaļ |
|
Akinori MUSHA
|
fca8051e81 |
Add a parser type `text` to WebsiteAgent. |
10 gadi atpakaļ |
|
Akinori MUSHA
|
d006fd662b |
Show user the correct default User-Agent value. |
10 gadi atpakaļ |
|
Akinori MUSHA
|
fa0723ed8a |
Correct an XPath in description. |
10 gadi atpakaļ |
|
Akinori MUSHA
|
7b6119f1f2 |
`"text": true` should have meant ".//text()", not "text()". |
10 gadi atpakaļ |
|
Akinori MUSHA
|
76ecced268 |
WebsiteAgent: Improve readability with regard to XPath evaluation. |
10 gadi atpakaļ |
|
Akinori MUSHA
|
4d623c5893 |
WebsiteAgent: Introduce per-node XPath evaluation in extraction. |
10 gadi atpakaļ |
|
Andrew Cantino
|
f4df522f2f |
adding a basic RSS agent |
10 gadi atpakaļ |
 Theofilos Papapanagiotou
|
669b77ce37 |
Ignore xmlns when evaluating xpath |
10 gadi atpakaļ |
|
Andrew Cantino
|
073df69df7 |
take dsander's advice and use interpolated instead of interpolated_options |
10 gadi atpakaļ |
|
Andrew Cantino
|
a8f32c5e85 |
Add memoized interpolated_options and used it everywhere. |
10 gadi atpakaļ |
|
Andrew Cantino
|
fd2e8cd8b6 |
add headers option to WebsiteAgent |
10 gadi atpakaļ |
|
Akinori MUSHA
|
e8751af629 |
Add a `user_agent` option to WebsiteAgent. |
10 gadi atpakaļ |
|
Akinori MUSHA
|
85a7369e65 |
Use Faraday in WebsiteAgent and make HTTP backend library selectable. |
10 gadi atpakaļ |
|
Akinori MUSHA
|
88e3132897 |
JSON does not allow the use of single quotation, so do not encourage it. |
10 gadi atpakaļ |
 Maximilian Clarke
|
23356e2a04 |
Added http/s check and cleaned up receive code |
10 gadi atpakaļ |
|
Maximilian Clarke
|
9bf3c2c824 |
Updated WebsiteAgent to receive events |
10 gadi atpakaļ |
|
Maximilian Clarke
|
0e30b083b4 |
Updated agent description |
10 gadi atpakaļ |
|
Maximilian Clarke
|
19c005fe45 |
Modified website_agent to take an array of urls |
10 gadi atpakaļ |
 Andrew Cantino
|
7d9279b871 |
Merge pull request #212 from knu/website_agent-force_encoding |
10 gadi atpakaļ |
|
Akinori MUSHA
|
df16d8dead |
Ignore force_encoding if empty. |
10 gadi atpakaļ |
|
Akinori MUSHA
|
8ea2ba573f |
Add :xpath support to WebsiteAgent. |
10 gadi atpakaļ |
|
Akinori MUSHA
|
7bc20a0b44 |
Add :force_encoding support to WebsiteAgent. |
10 gadi atpakaļ |
|
Andrew Cantino
|
99644a426d |
Add XKCD hovertext to default seed and website agent |
10 gadi atpakaļ |
|
Andrew Cantino
|
0ad347c378 |
default is all |
10 gadi atpakaļ |
|
Andrew Cantino
|
f4bae10250 |
minor code cleanup |
10 gadi atpakaļ |
 Alex Piggott
|
3f4e4b7531 |
#154 Fixed indentation (website deduplication improvements) |
10 gadi atpakaļ |
|
Alex Piggott
|
b1898cc7ff |
#154 Improvements to website deduplication logic |
10 gadi atpakaļ |
|
Alex Piggott
|
7b38df61ed |
#135 #141 2 deduplication fixes for the website agent |
10 gadi atpakaļ |
 Albert Sun
|
18595b0760 |
blank? -> present? |
10 gadi atpakaļ |
|
Albert Sun
|
7996954a3b |
add a `basic_auth` option to the website agent |
10 gadi atpakaļ |
|
Andrew Cantino
|
00b7423dd7 |
add cached columns for event creation and last errors, reducing the number of SQL queries |
10 gadi atpakaļ |
|
Andrew Cantino
|
a408ae48ea |
globally avoid using symbols since we're moving to json storage |
10 gadi atpakaļ |
|
Albert Sun
|
f0741359a6 |
DRYed up website_agent code a bit |
11 gadi atpakaļ |
|
Albert Sun
|
43194c3c1b |
in website agent with type json, allow extract to be blank; in which case, the entire json object will be stored as the payload |
11 gadi atpakaļ |
|
Andrew Cantino
|
7372244d0f |
return false from working? when an agent's most recent log is an error |
11 gadi atpakaļ |
|
Andrew Cantino
|
bdea7e620c |
log all exceptions during check or receive to Agents' logs |
11 gadi atpakaļ |
|
Andrew Cantino
|
00727fbd4d |
add Agent Logs; add logging to WebsiteAgent; refactor flash notices and add event notices |
11 gadi atpakaļ |
 Andrew Cantino
|
86b5b1f203 |
Use select2 for type selection; switch to showing JSON for event information |
11 gadi atpakaļ |
 itkevin
|
430a6a8432 |
Code cleanup |
11 gadi atpakaļ |
 itkevin
|
fd8761177f |
When crawling websites tith relative URLs, make them absolute |
11 gadi atpakaļ |
Andrew Cantino
Akinori MUSHA
Theofilos Papapanagiotou
Maximilian Clarke
Andrew Cantino
Alex Piggott
Albert Sun
Andrew Cantino
itkevin
itkevin