Web scrapping

Process of extracting data from web pages for use in another program. Web scrapping allows to create custom API for almost any web resource.

Notes

Some useful tips:

• Use Google cache version of target site until developing a scrapper

  http://webcache.googleusercontent.com/search?q=cache:<TARGET_URL>

• Use headless browser

• Use “Referer”: “https://www.google.com/” header

Links

• 5 Tips For Web Scraping Without Getting Blocked or Blacklisted