If anyone would like to follow some of the same journals, I've uploaded them to my site as an opml, which you can import into most RSS readers. h/t to @dgold for the suggestion.
Download it here: http://lrhodes.net/assets/files/open_access_academic_journals.opml.xml
These are specifically feeds for open access articles, so you shouldn't hit any paywalls. And if you have any suggestions for other journals that have an RSS feed for open access articles, do let me know!