Custom Built Google Extension for Manual Scraping

Developing automatic website scrapers can sometimes be challenging due to how modern websites are built. Some of these challenges are the continuous layout changes, captchas, bot patterns, paginations, being anonymous, logins and websites that populate new content by continuously scrolling.

I was tasked with developing a Google Extension that would help with speeding up manual scraping. The biggest benefit of developing a plugin is that the user can load a website as if they are browsing it normally which reduces all the challenges automated scraping is faced with.

The plugin was built in such a way that it allows the user to click anywhere in the page and automatically extract the text. The text is then passed through to a system which is then translated, cleaned and mapped to certain fields in the system. A summary is also visible showing the scraping progress.

Like what you see then, fancy a chat?

Email me Phone me

Testimonials

Dean has recently completed our Kitchens, Bedrooms and Bathrooms website and we are absolutely delighted with the result. We asked for a website that would be uncluttered, clean and simple in design and Dean has interpreted this perfectly. Dean’s gentle guidance throughout the process has been fantastic, he is very easy to communicate with at very flexible times of the day with quick response to questions, queries and changes.  I would be more than happy to recommend Dean to anyone seeking a web-site design service.

Helen Lamont - Complete Kitchens, Holsworthy