Automated monitoring of web pages using Page2RSS, Feedly and IFTTT

Automated monitoring of web pages using Page2RSS, Feedly and IFTTT

So you are an activist trying to keep a pulse on the community you’re serving. Unfortunately, you have little time and a lot of web pages, groups, and updates to keep track of. Luckily, by using a few simple tools, you can automate away much of your busy work, leaving you more time to tackle the hard stuff.

This post will show you how to use these tools to automatically monitor any web page to notify you when there are changes you might be interested in.

The steps to monitor any page are:

  1. Identify what changes on the page, and what keywords always are present during the change
  2. Create a Page2RSS for that page
  3. If simple enough, subscribe with Feedly. If not, create an IFTTT that filters based on one of the keywords and send to Feedly.

At step 3, you can also email yourself with a change, post to twitter, etc- anything that you can do in IFTTT.

Quick question, Colin: what does all that jargon mean???

Yes, there are some new terms here. Let’s make a quick glossary:

RSS feed: a way to organize updates to websites in easily-automatable and simple, bite-sized chunks

Page2RSS: An awesome web service that checks in on a page every now and again and packages the difference between the versions of the website into an RSS feed

Feedly: A service like (the now dead) Google Reader which you can use to collect items from RSS feeds. It takes care of reading the RSS feed and presenting the update in a human-readable form.

IFTTT: “If this then that”, a simple service that can hook together parts of the internet to make you a super-activist. It is worth learning this tool very well.

What you’ll need:

  • An IFTTT account, with the ‘Email’ and ‘Feedly’ channels activated
  • A Feedly account or an email account
  • A Page2RSS account (optional, but useful to organize)

Workflow:

page notify workflow
Three different options to get notified about changes

The examples below follow these three tracks. The simplest is subscribing to all changes to the page using Feedly, while the most complicated is using IFTTT to filter out only the changes you want to be notified by. Don’t worry, once you get the hang of things it’s really very simple!

Examples:

Herman, MO – simple any change to page and email:

In this case, we are just checking for any change to the city council page, and we want an email any time it changes. This represents the  yellow track above.

Here’s the Herman, MO page:

Herman, MO City Council Page
Herman, MO City Council Page

We want to be notified of any change to this page. Let’s first turn the page into an RSS feed, using Page2RSS:

create rss monitor
Paste in the URL of the page you want to monitor here, or use the Chrome extension

(If you install the Chrome extension you can cut down your manual work even more – log into Page2RSS and download the extension to boost your productivity!)

Herman City Council to RSS
Turning the city council page into an RSS feed

Now, if we want to just add this to our Feedly stream, we can just click on the ‘Feedly’ button on that page and we’re done! This is the purple track in the workflow above.

However, for this example, we want an email every time this page changes. Click on the ‘RSS’ button and copy the url:

herman rss feed url
Copy the URL here

Now head over to IFTTT. Let’s create a new ‘recipe’ choosing the RSS ‘channel’. The icon looks like this:

rss icon
Look for this icon when choosing a trigger ‘Channel’

We can paste the URL saved from Page2RSS into the creation box:

add rss trigger in IFTTT
Paste the RSS feed URL from Page2RSS
IFTTT Choose Action
What you should see after creating the RSS Feed trigger

IFTTT is a tool that allows you to automatically take action without human intervention. You can specify different ‘Triggers’ which then kick off ‘Actions’. If <Trigger happens> then <Take action>. “If this, then that”. In our case: “If the RSS feed is updated, email me”. So choose the ‘Email’ Channel for our ‘Action’. You will then see:

choose email
After choosing the ’email’ channel
edit email action
Edit this to suit your preferences

There will be some confusing text already entered into this page for you. Don’t delete what’s here but:

  1. Add a subject that makes sense to you before the “EntryTitle” text in the subject line. In this case, I added “Herman, MO City Council”
  2. Make a link directly to the city council page in the email- this will save you time later

Click ‘Create Action’, and you’re done! Whenever that page changes, you’ll get an email. This is the yellow track in the workflow above.

Columbia, MO – filter by ‘Minutes’ and post to Feedly:

I want to know when the minutes from the latest city council meeting of Columbia, MO, are updated. In this example, I’ll show how to only alert when the page changes in a specific way. I will also have the update post to Feedly instead of emailing. This is the red track in the workflow above.

Let’s take a look at the city of Columbia, MO:

Columbia, MO City Council
Columbia, MO city council page

We can see that there are three likely items that change on the page on a regular basis:

  1. New meeting dates (e.g. December 16, 2013 | Columbia City Council Regular Meeting)
  2. Agendas are posted, as a link with the word “Agenda”
  3. Minutes are posted, as a link with the word “Minutes”

Let’s say we only care about when minutes are posted, so we don’t want to just subscribe to any changes on the page.

First we grab the URL and go to page2rss.com:

page2rss
Get page2rss link from council page

Press the ‘to RSS’ button, and you should see:

Page2RSS Results
Page2RSS Results

Great. If we wanted to just subscribe to this feed, we could click on the “Feedly” button on this page (the purple track). However, in this case we don’t want an update when Agendas or future events are posted, so we need to grab the RSS link and head over to IFTTT. You can press the ‘RSS’ button in the “Subscribe to Feed” section and copy the link:

Page2RSS RSS link
Page2RSS RSS link

Head over to IFTTT.com and create a new ‘Recipe’ using the RSS ‘Channel’:

IFTTT create
Create a new recipe in IFTTT

Choose “New feed item matches”

Filter RSS in IFTTT
Filter RSS match in IFTTT

We’ve completed the ‘this’ portion of IFTTT, now we need to make the ‘that’ action:

IFTTT Choose Action
Choose the ‘then’ action. Any ‘Channel’ is possible.

Let’s choose to publish to Feedly, although you could also have it send you an email you, tweet at you, text you, whatever you want!

Choose your IFTTT Action
Choose your IFTTT Action

Choose “Add new source”, and perhaps edit the URL to make it easier for you to read. In this case I added ” – Columbia City Council – Minutes”:

Change title of Feedly URL in IFTTT
Change the title of Feedly URL in IFTTT to be easier for you to understand

Finally, accept the new recipe, again perhaps changing the ‘description’ field to be a little more human-readable.

Accept IFTTT recipe
Accept IFTTT recipe

That’s it! Whenever the Columbia, MO city council updates their minutes, a post will appear in your Feedly stream.

You now only need to look at your Feedly page to see which city councils have updated their minutes- no more clicking though to each town and scanning the site to see if there’s been an update.

Conclusion

Using Page2RSS, IFTTT, and Feedly/Email, we can automate away some of the most tedious parts of activism. This helps you stay on top of any situation more quickly, and saves you time to work on more important things. Share this with your fellow activists- let’s use technology to improve our reach for the same amount of effort!

If you have any questions, please comment below- I’d love to have feedback on how to improve this guide.

6 thoughts on “Automated monitoring of web pages using Page2RSS, Feedly and IFTTT

  1. Great article, just saved me hours and hours of time. Do you know of any other solid sites that monitor pages and send back JSON or XML with the element changed?

    Thanks a ton,
    J

  2. Nice, glad you liked it!
    I think Ronno’s comment is great – that looks like a better site than page2rss actually…

  3. One good service which is even for free is https://www.wachete.com
    You can monitor part of page, whole page or crawl whole page with subpages.
    Notifications via email when page changes or any other condition you set.
    Cool is also that it comes with mobile apps for all platforms.

Leave a Reply

Your email address will not be published.