SM Sith Lord

Week 10

This past week the web data scraping system has continued to be expanded and enhanced.  I implemented a way for a scraper to specify how confident it is that it has found quality content for each of the item’s fields.  I also implemented a way for a scraper to suggest yet another scraper be ran afterwards to fill in fields that still need content.
For example, lets say that you’re watching a Netflix movie and click the Spawn Now button, first the Netflix scraper runs and grabs basic info about the movie like the title, the URL, and a DVD cover, but now the NetFlix scraper will also suggest that the TheMovieDb.org’s scraper be ran next to find additional content.
So then TheMovieDb.org is searched for the title of the movie.  If it is found, then the TheMovieDb.org scraper runs on it and grabs YouTube trailers and HD wallpapers to complete the item.
All of this is completely automated, JavaScript-powered, and implemented as independent addon scripts for each scraper.
Anarchy Arcade’s ability to automatically decorate a 3D shortcut with high-quality images and videos is what makes it such an immersive, media-rich experience.
Since data scrapers are such a vital part of Anarchy Arcade, I will continue working on them this week.  The existing scrapers for Netflix and TheMovieDb will be cleaned up & commented so that I can show them off the following week in the development blog post.  They will be shown off as fully functional Google Chrome extensions.


Posted

in

,

by

Tags: