🌱 Digital garden
  • Welcome
  • Courses
    • Fundamentals of 2D Game Engines with C++ SDL and Lua
      • Dependencies and project structure
  • Computer science
    • Basic
  • Sound
    • FM synthesis
  • Development
    • Design system
    • Text editors
      • Vim
      • Emacs
      • VS Code
    • Databases
      • MongoDB
    • NES Development
    • Programming languages
      • Shell
      • Javascript
        • Arrays
        • JSDoc
    • UML
    • Web development
      • HTTP
      • GraphQL
      • PWA
      • CSS
      • REST
      • Web scrapping
      • React
        • React hooks
      • HTML5
        • Canvas
          • WebGL
          • Context2D
    • Game development
      • Pixel art
      • AABB collision
      • Sprite
      • Game loop
  • Education
    • Tools
      • Anki
  • Electronic
    • Components
      • Opto-isolator
  • Image quality
  • Languages
    • English
  • Utilities
    • Docker
    • SSH
  • Lifestyle
    • Minimalism
    • Reading
  • Science
    • Math
      • Fractions
  • Productivity
    • Bullet journaling
    • Typing
  • Operating system
    • Unix
    • Linux
    • MacOS
  • Learning resources
  • Research
Powered by GitBook
On this page
  • Notes
  • Links

Was this helpful?

  1. Development
  2. Web development

Web scrapping

Process of extracting data from web pages for use in another program. Web scrapping allows to create custom API for almost any web resource.

Notes

Some useful tips:

  • Use Google cache version of target site until developing a scrapper

    http://webcache.googleusercontent.com/search?q=cache:<TARGET_URL>

  • Use headless browser

  • Use “Referer”: “https://www.google.com/” header

Links

  • 5 Tips For Web Scraping Without Getting Blocked or Blacklisted

PreviousRESTNextReact

Last updated 4 years ago

Was this helpful?