Open Source R Internet Software

rvest helps you scrape (or harvest) data from web pages. It is designed to work with magrittr to make it easy to express common web scraping tasks, inspired by libraries like beautiful soup and RoboBrowser. If you’re scraping multiple pages, I highly recommend using rvest in concert with polite. The polite package ensures that you’re respecting the robots.txt and not hammering the site with too many requests.

httr is superseded: only changes necessary to keep it on CRAN will be made. We recommend using httr2 instead. The aim of httr is to provide a wrapper for the curl package, customized to the demands of modern web APIs. Functions for the most important http verbs: GET(), HEAD(), PATCH(), PUT(), DELETE() and POST(). Automatic connection sharing across requests to the same website (by default, curl handles are managed automatically), cookies are maintained across requests, and an up-to-date root-level SSL certificate store is used. Requests return a standard reponse object that captures the http status line, headers and body, along with other useful information. Support for OAuth 1.0 and 2.0 with oauth1.0_token() and oauth2.0_token(). The demo directory has eight OAuth demos: four for 1.0 (twitter, vimeo, withings and yahoo) and four for 2.0 (facebook, github, google, linkedin). OAuth credentials are automatically cached within a project.

YA BASTA! is a Python/R application for Lyrics Web Scraper and Text Analysis. Web scraping is developed in Python, text analysis in R as Python subprocesses. YA BASTA! is only tested on windows OS. To run YA BASTA! just type on window command prompt: python.exe yabasta.py