Automated monitoring of web pages using Page2RSS, Feedly and IFTTT
So you are an activist trying to keep a pulse on the community you’re serving. Unfortunately, you have little time and a lot of web pages, groups, and updates to keep track of. Luckily, by using a few simple tools, you can automate away much of your busy work, leaving you more time to tackle the hard stuff.
This post will show you how to use these tools to automatically monitor any web page to notify you when there are changes you might be interested in.
The steps to monitor any page are:
- Identify what changes on the page, and what keywords always are present during the change
- Create a Page2RSS for that page
- If simple enough, subscribe with Feedly. If not, create an IFTTT that filters based on one of the keywords and send to Feedly.
At step 3, you can also email yourself with a change, post to twitter, etc- anything that you can do in IFTTT.
Quick question, Colin: what does all that jargon mean???
Yes, there are some new terms here. Let’s make a quick glossary:
RSS feed: a way to organize updates to websites in easily-automatable and simple, bite-sized chunks
Page2RSS: An awesome web service that checks in on a page every now and again and packages the difference between the versions of the website into an RSS feed
Feedly: A service like (the now dead) Google Reader which you can use to collect items from RSS feeds. It takes care of reading the RSS feed and presenting the update in a human-readable form.
IFTTT: “If this then that”, a simple service that can hook together parts of the internet to make you a super-activist. It is worth learning this tool very well.
What you’ll need:
- An IFTTT account, with the ‘Email’ and ‘Feedly’ channels activated
- A Feedly account or an email account
- A Page2RSS account (optional, but useful to organize)
Workflow:
The examples below follow these three tracks. The simplest is subscribing to all changes to the page using Feedly, while the most complicated is using IFTTT to filter out only the changes you want to be notified by. Don’t worry, once you get the hang of things it’s really very simple!
Examples:
Herman, MO – simple any change to page and email:
In this case, we are just checking for any change to the city council page, and we want an email any time it changes. This represents the yellow track above.
Here’s the Herman, MO page:
We want to be notified of any change to this page. Let’s first turn the page into an RSS feed, using Page2RSS:
(If you install the Chrome extension you can cut down your manual work even more – log into Page2RSS and download the extension to boost your productivity!)
Now, if we want to just add this to our Feedly stream, we can just click on the ‘Feedly’ button on that page and we’re done! This is the purple track in the workflow above.
However, for this example, we want an email every time this page changes. Click on the ‘RSS’ button and copy the url:
Now head over to IFTTT. Let’s create a new ‘recipe’ choosing the RSS ‘channel’. The icon looks like this:
We can paste the URL saved from Page2RSS into the creation box:
IFTTT is a tool that allows you to automatically take action without human intervention. You can specify different ‘Triggers’ which then kick off ‘Actions’. If <Trigger happens> then <Take action>. “If this, then that”. In our case: “If the RSS feed is updated, email me”. So choose the ‘Email’ Channel for our ‘Action’. You will then see:
There will be some confusing text already entered into this page for you. Don’t delete what’s here but:
- Add a subject that makes sense to you before the “EntryTitle” text in the subject line. In this case, I added “Herman, MO City Council”
- Make a link directly to the city council page in the email- this will save you time later
Click ‘Create Action’, and you’re done! Whenever that page changes, you’ll get an email. This is the yellow track in the workflow above.
Columbia, MO – filter by ‘Minutes’ and post to Feedly:
I want to know when the minutes from the latest city council meeting of Columbia, MO, are updated. In this example, I’ll show how to only alert when the page changes in a specific way. I will also have the update post to Feedly instead of emailing. This is the red track in the workflow above.
Let’s take a look at the city of Columbia, MO:
We can see that there are three likely items that change on the page on a regular basis:
- New meeting dates (e.g. December 16, 2013 | Columbia City Council Regular Meeting)
- Agendas are posted, as a link with the word “Agenda”
- Minutes are posted, as a link with the word “Minutes”
Let’s say we only care about when minutes are posted, so we don’t want to just subscribe to any changes on the page.
First we grab the URL and go to page2rss.com:
Press the ‘to RSS’ button, and you should see:
Great. If we wanted to just subscribe to this feed, we could click on the “Feedly” button on this page (the purple track). However, in this case we don’t want an update when Agendas or future events are posted, so we need to grab the RSS link and head over to IFTTT. You can press the ‘RSS’ button in the “Subscribe to Feed” section and copy the link:
Head over to IFTTT.com and create a new ‘Recipe’ using the RSS ‘Channel’:
Choose “New feed item matches”
We’ve completed the ‘this’ portion of IFTTT, now we need to make the ‘that’ action:
Let’s choose to publish to Feedly, although you could also have it send you an email you, tweet at you, text you, whatever you want!
Choose “Add new source”, and perhaps edit the URL to make it easier for you to read. In this case I added ” – Columbia City Council – Minutes”:
Finally, accept the new recipe, again perhaps changing the ‘description’ field to be a little more human-readable.
That’s it! Whenever the Columbia, MO city council updates their minutes, a post will appear in your Feedly stream.
You now only need to look at your Feedly page to see which city councils have updated their minutes- no more clicking though to each town and scanning the site to see if there’s been an update.
Conclusion
Using Page2RSS, IFTTT, and Feedly/Email, we can automate away some of the most tedious parts of activism. This helps you stay on top of any situation more quickly, and saves you time to work on more important things. Share this with your fellow activists- let’s use technology to improve our reach for the same amount of effort!
If you have any questions, please comment below- I’d love to have feedback on how to improve this guide.
6 thoughts on “Automated monitoring of web pages using Page2RSS, Feedly and IFTTT”
you may also use http://bitreading.com/deltafeed for the page 2 rss functionality…
Great article, just saved me hours and hours of time. Do you know of any other solid sites that monitor pages and send back JSON or XML with the element changed?
Thanks a ton,
J
Nice, glad you liked it!
I think Ronno’s comment is great – that looks like a better site than page2rss actually…
One good service which is even for free is https://www.wachete.com
You can monitor part of page, whole page or crawl whole page with subpages.
Notifications via email when page changes or any other condition you set.
Cool is also that it comes with mobile apps for all platforms.
A site that I find pretty easy to use for monitoring web pages is https://pagedelta.com . I like how it emails me a screenshot of what changed.