Week 10

This past week the web data scraping system has continued to be expanded and enhanced.  I implemented a way for a scraper to specify how confident it is that it has found quality content for each of the item’s fields.  I also implemented a way for a scraper to suggest yet another scraper be ran afterwards to fill in fields that still need content.
For example, lets say that you’re watching a Netflix movie and click the Spawn Now button, first the Netflix scraper runs and grabs basic info about the movie like the title, the URL, and a DVD cover, but now the NetFlix scraper will also suggest that the TheMovieDb.org’s scraper be ran next to find additional content.
So then TheMovieDb.org is searched for the title of the movie.  If it is found, then the TheMovieDb.org scraper runs on it and grabs YouTube trailers and HD wallpapers to complete the item.
All of this is completely automated, JavaScript-powered, and implemented as independent addon scripts for each scraper.
Anarchy Arcade’s ability to automatically decorate a 3D shortcut with high-quality images and videos is what makes it such an immersive, media-rich experience.
Since data scrapers are such a vital part of Anarchy Arcade, I will continue working on them this week.  The existing scrapers for Netflix and TheMovieDb will be cleaned up & commented so that I can show them off the following week in the development blog post.  They will be shown off as fully functional Google Chrome extensions.

Week 9

This week I started towards the goal of getting a working “Create New Item” system into the Steam version of Anarchy Arcade.  A very important part of the Create New Item system is AArcade’s ability to go onto the web and automatically find boxart, wallpapers, and video trailers for the media items that are spawned in it.  This was done in the prototype by the “scrapers.”  In the Steam version, the entire system has been overhauled and expanded into something called Providers.  These changes were necessary to enhance security, expand functionality, and allow user-created addons to be more coherently organized.

A Provider provides many important web-services to the player.  Each provider is usually related to one specific online website or service, such as YouTube or Netflix.  It is written primarily in JavaScript and it may have up to six different functions; however, not every provider needs to have all 6 functions.

Providers are user-created addons, and therefore could possibly be from malicious users.  AArcade carefully loads and executes these six functions only at specific times to provide maximum security.  Certain operations require more security sandboxing than others, and organizing the Providers into these six specific functions helps manage this.

The six functions that a provider may perform are: safeTest, test, scrape, autoScrape, search, and autoSearch.  I will very briefly try to explain what each one of these functions do.  Keep in mind that security was one of the primary concerns when I designed these six functions.  Only providers that you trust should be allowed to perform any of the functions other than safeTest.

In general, a provider is used when ever you click the SPAWN button while looking at a website in the in-game browser.  Some features of the provider, such as search, might also be used when spawning new local files from your hard drive too.

safeTest
The provider analyzes the current URL string of the in-game browser and tries to determine if it can be used on the page or not.  This is a nearly risk-free function because the provider is only given a URL to analyze.  However, it is not always possible to determine if a provider can be used merely by analyzing the URL.

test
The provider analyzes the live webpage that is currently loaded in the in-game browser to determine if it can be used on the page or not.  The provider has full access to the webpage’s JavaScript and DOM.  This allows elements or variables that were created even after the page was initially loaded to be analyzed by the provider to determine if it can be used.

scrape
The provider analyzes the live webpage that is currently loaded in the in-game browser and attempts to construct a complete AArcade Item profile from the information on the page.  The provider has full access to the webpage’s JavaScript and DOM.  The provider is able to prompt the player to choose options, if needed.

autoScrape
The provider analyzes a read-only copy of a webpage’s DOM and attempts to construct a complete AArcade Item profile from the information on the page.  The provider does not have access to any live JavaScript functions or variables, but only the information contained in the static DOM.  This allows for more automation and better security than the powerful scrape function discussed above.

search
The provider loads a live search results page for a specific search term into the in-game browser and analyzes the results to automatically find a match to the media item the player is spawning.  The provider has full access to the webpage’s JavaScript and DOM.  The provider is able to prompt the player to choose options, if needed.

autoSearch
The provider loads a read-only copy of a search results page for a specific search term and analyzes the results to automatically find a match to the media item the player is spawning.  The provider does not have access to any live JavaScript functions or variables, but only the information contained in the static DOM.  Once again, this allows for more automation and better security.

Using these six functions, the Steam version of Anarchy Arcade will be able to offer all the amazing features showcased in the prototype, PLUS many great enhancements & additions, all as user-addon scripts.

Separating the functionality of Anarchy Arcade from the content and services that are used in it has always been the goal of its design.  Providers are a major step towards this goal because they change AArcade from supporting only a few services such as “YouTube & Netflix” to supporting EVERYTHING.  It also makes AArcade more versatile because it is able to out-live the services used in it.  Sure, MySpace is hot stuff now, but one day it might become old news!

AArcade Providers as Chrome Extensions
The Chromium Embedded Framework (CEF) is what powers AArcade behind-the-scenes.  I’ve never used Google Chrome before, so last week it was a pleasant surprise when I learned that extensions for the Google Chrome browser are also written in JavaScript, just like the data-scrapers used in AArcade!  This means I am able to prototype the data-scrapers and the data-scraping system itself as Chrome extensions.

So last week I learned how to write Chrome extensions, and ported the Netflix scraper from the prototype over into the new Provider design.  It’s working great so far!  However, right now it’s all just a bunch of loose JavaScript files.  Like Detective Murphy in the Robocop re-make; just a bunch of vital organs floating around in jars plugged in to each other with tubes.  There is still a lot of work to be done on these organs before I’m ready to build the rib-cage that holds them.  Next week I will be refining the JavaScript implementations of the providers design so that it can play nicely with the other systems I have been working on.

Google’s JPEG
So, there’s a new image format in town.  Apparently it’s been around for a while, but I haven’t noticed yet because I never used Google Chrome.  It’s called .webp and it is created by the tech power-house Google.  It has superior compression than JPEG files and saves tons of bandwidth which is extremely important on mobile phones, so Google has been pushing it a lot.  Problem is, lots of browsers and programs don’t support this .webp image format because they feel it is either too new, too crappy, or too Google.  The result is some major websites, such as Facebook and Netflix, serving you .webp images only if you’re on a browser that supports it, but sending normal JPEG images otherwise.

This is unfortunate, to say the least.  Until it is widely adopted, it means that when you link your friend to an image, it might be an image that requires a specific web browser to view.

Prototype users will have noticed the Netflix spawner is currently broken.  This is because the image loader in the prototype does not support .webp images and causes the issue.  While a patch will be released soon for the prototype to get the Netflix spawner working, the .webp image loading issue will not be addressed until the Steam version.

This is because the Steam version of AArcade uses the Chromium Embedded Framework itself as the image loader, and therefore can properly load .webp images natively.

Joypads & Web Browsing
This week I also wrote most of the code that allows you to browse the web in the in-game browser using your joystick!  Not only will you navigate websites with this input system, but also the UI menus of AArcade itself because they are HTML/JavaScript powered!

Also, some core-systems were worked into the in-game browser that will make things a lot easier to do the things you want.  You will finally be able to right-click on specific elements of webpages.  Drop-down menus will now display properly, and websites can now display prompts & alerts.

Once again, these features are just like Robocop’s vital organs floating in jars.  Their code is functioning independently, but the body still needs to be built to hold them all together.

Coming Up
This week I will continue refining the providers system (which is written mostly in JavaScript) and then start building the back-end for the providers into the AArcade core (written in C++).

Week 8

Hello fellow arcade-goers! This week of Anarchy Arcade development was spent prototyping the all-new Inventory App (formally known as the Library Browser) that gets displayed on your in-game virtual phone and acts as the HTML-powered user interface to access all of your media on your PC and on the world-wide web.

Now, it’s time to get a little technical.  If you’re not interested in coding or design, feel free to skip the next few paragraphs. :)

The first step of creating the Inventory App for the in-game virtual phone was to create a new kind of interactive HTML/JavaScript object that I could place inside of a .html page.  It would provide all of the functionality needed to send search requests to the C++ back-end of Anarchy Arcade, and then display the search results in a useful interface that the player could easily sort and customize the look of.

Developing the UI is very fun for a couple of reasons.  The internal Anarchy Arcade UI is powered by the Chromium web browser framework, and the API for accessing the Anarchy Arcade back-end is accessible via HTML/JavaScript.  This means that I can do a large amount of development for the in-game virtual phone apps using a text editor and the standalone version of the Chrome web browser.

To make this kind of development possible, all I have to do is write mock-up return values for any calls that my in-page JavaScript makes to the AArcade core.  Then, after the front-end is all finished, I just code the actual functions into the real AArcade API so that the values being returned to the front-end are real search results from the user’s library instead of mock-up values.

It’s too early to make the full version of the Inventory App, so this week it was all about getting a good prototype of the app so that I could adjust the AArcade JavaScript API accordingly to be robust enough to handle things like the Inventory page and the Create New Shortcut page.

You can check out a demo of this new Inventory HTML/JavaScript object here: http://smsithlord.com/library_search.html

Now that the functionality of the Inventory app is worked into the AArcade JavaScript API, it’s time to move on to the Create New Item functionality.  This will be the focus of the next week of development and I will have details about what it is and how it went in 7 days.  See you then.

Don’t forget, every Wednesday, there’s a 24 hour www.twitch.tv/AnarchyArcade live video stream if you have any questions.

Week 7

This week I optimized the back-end AArcade JavaScript API and the thumbnail image loading system.  The image loading system is just one of the many systems that use the AArcade JavaScript API to power their functionality.  Making sure that the Chromium core and the AArcade JavaScript API are robust enough to handle everything that they are going to be used for is vital at this stage of development to ensure that all goes smoothly later on.

There are several different locations that each 3D shortcut might use to load artwork images to use to decorate its in-game cabinet.  These include web-based HTTP addresses, or local image file locations.  One of the new features worked into Anarchy Arcade’s brand new image loading system this week was the ability to detect when an image fails to load, and attempt to use alternate image locations accordingly.

Even at this early stage, everything that is implemented into Anarchy Arcade has multiplayer in mind.  This past week I tied the image loading system into the multiplayer item syncing system, so now the images that are being loaded and displayed on the cabinet models are calculated directly from the attributes of the 3D shortcuts themselves.  This is a huge step towards getting the Steam version of Anarchy Arcade to the type of functionality seen in the prototype.

All of the work over the past few weeks has been towards the ultimate goal of being able to implement the data scraping system that powers all of the behind-the-scenes work that goes into turning a boring standard desktop shortcut into a HD media-rich 3D shortcut.  It is also what powers all of the 3D Web features of Anarchy Arcade and is vital for implementing HUBs.  The data scraping features of Anarchy Arcade are responsible for a majority of the automation that is seen in the prototype as well as the amazing new features built into the Steam version’s design.

The web data scraping system will be built on top of the AArcade JavaScript API, so next week as I continue to implement the core functionality of Anarchy Arcade I will be programming in the foundation of the web data scraping system and expanding the AArcade JavaScript API.  All of this will be done in such a way to accommodate HUBs, multiplayer, and all of the other systems that are deeply connected to how web data scraping will behave.

On Wednesday, I will be hosting a live video stream using the prototype (which is still publicly supported even though it does not receive updates anymore).  There will be live chat too, so if you have any questions about Anarchy Arcade or the prototype, be sure to “tune-in” to www.twitch.tv/AnarchyArcade on Wednesdays between noon and midnight to participate.

Weeks 5 & 6

After a lot of studying my own flowcharts, notes, and code from a few months ago, I am back up to speed with development.  As you know, the Steam version is a completely new build of Anarchy Arcade that is written from the ground-up.  Right now I’m pounding away at back-end features, so there’s not a lot to show visually yet.

This week started off with me making flowcharts and modifying UML diagrams to figure out the details of certain things, like hubs, which are basically a group of many different shortcuts that you can spawn as a single unit.

However, instead of always displaying the same shortcuts, these hubs can be dynamic so that they always show the newest YouTube videos from a specific channel, or always the highest rated content from your favorite website.

These hubs are controlled by JavaScripts that allows for limitless customization and addons hubs to be created and shared with everyone.  The implementation of these hubs is very powerful and will allow for lots of automation in the Steam version of Anarchy Arcade.

For example, instead of having to individually spawn 1 shortcut at a time to populate your house, each room will have a variety of pre-set locations for your media to spawn at.  You’ll be able to cycle through the different presets and choose from many different room arrangements that automatically populate with your own games, movies, and media in an intelligent way.  Of course, you can still spawn your media in one at a time if you wish.

All of that work was done earlier this week with flowcharts and UML.  Later in the week I was able to actually get my hands dirty in the code, REAL dirty.

My programming objective was to get images & websites loading into textures that are displayed on each inactive shortcut.  In the prototype, EVERY texture is loaded 1 at a time and gets resized to a standard image size of 512×512, but this time around I was able to get something much nicer worked in.  However, getting images & websites to load into Source engine as VTF textures required quite a bit of work and cooperation between different systems.

I have begun implementing the 2-way JavaScript AArcadeAPI which allows the Chromium-powered UI (which is essentially like a web page) to communicate effectively with the Anarchy Arcade game core.

Chromium, which I use to power the UI, is also used to convert images & websites into Source engine VTF textures.  Web images sometimes take a long time to load, so now multiple images can be downloading simultaneously.

Instead or resizing images to a standard size, such as 512×512 for the prototype, the Steam version of Anarchy Arcade is able to load all images at within 4 pixels of their original image size.  What does that mean?  First, it means no more wasted memory scaling small images up to 512×512.  Second, it means that large resolution images will appear as high-detail images in-game (even on inactive cabinets).  And third, it means that the aspect ratio of the image will not be lost, which will help with the “stretched/squashed screen” issue that was in the prototype.

I’m happy to report that last night I was finally able to plug all the different systems in together and render images AND entire websites onto the in-game cabinet screens.

This 2-way JavaScript AArcadeAPI, the Chromium core, and the image/website thumbnail system are extremely important to all aspects of Anarchy Arcade so I will be spending next week streamlining them and preparing them for other features that are soon to come.

The prototype’s UI had stability and compatibility issues with certain websites.  I will be spending the next week making sure that the Steam version does not have these issues by optimizing the implementation at this early stage.

Week 4

The past two months I spent running/promoting the Kickstarter campaign with lots of news articles and prototype updates.  But now that the Kickstarter is concluded, it’s time for me to get back to coding right where I left off about two months ago.

I’m trying to get to the part where Anarchy Arcade will download your images from URLs again, but first I need to code in the part that resolves an image location to an actual URL.  This is because image location values can point to a few different types of things: local images, web images, or Steam community images.  I also must finalize the file structure that files will be saved into your AA folder with.

This means studying the code that I have implemented thus far into the Steam version (to jog my memory) and writing some fresh flow charts to cover all the different use-cases that might arise.

I’ll be streaming some development from time to time on twitch.tv/AnarchyArcade so be sure to follow my Twitch channel if you want to be notified when the stream goes live!  Also, every Wednesday the stream will be live all day long as I play video games instead of working on development. :)

Anarchy Arcade Development: Weeks 2 & 3

Things got off to a slow start as development was delayed by nearly 4 days due to a hardware failure, but now everything is up and running again.

Once again development was focused on revamping the backend to support all the new features that will be thrown its way.  Most of the work done this week was aimed at the in-game Smartphone and how the player will interact with AArcade.

All menus will be displayed to the player via the in-game Smartphone.  This will keep the player completely immersed in their virtual world and also avoid some potential issues when it comes time to implement the Occulus Rift.  Generally, in-game virtual objects are much more natural on the Occulus Rift than fullscreen menus.  By displaying the menus on the screen of your in-game Smartphone being held right in your character’s virtual hands the immersion will be complete.  Of course, you will still have the option to display the menus fullscreen if you prefer.

One issue that the community raised about the prototype concerns improper aspect ratios on the screens of certain cabinets.  This problem was due to the fact that the internal screen resolution of the prototype was 720p widescreen, while some cabinets used a display screen that was portrait instead of widescreen.

To resolve this issue, in the Steam version the internal resolution can be switched between widescreen and portrait on-the-fly.  This fixes the issue affecting certain cabinet screens, but also prepares the UI for the all-new virtual Smartphone Apps that can be displayed on your in-game Smartphone in either portrait or landscape mode; just like a real Smartphone!

The design objective over the next couple of weeks will be to implement the new & improved spawning and data scraping systems into the Steam version of Anarchy Arcade.  This is where many aspects of the design are going to start coming together and connecting.  Sometimes spawning shortcuts requires data scrapers and OpenWith profiles.  Sometimes data scrapers require user interaction.  User interaction requires the Smartphone.  The Smartphone requires its own virtual operating system complete with both official & addon Apps, as well as its own JavaScript API for communicating between the Smartphone and the AArcade backend.

The groundwork for all these features is already in-place and over the next couple of weeks both the user experience & functionality of Anarchy Arcade will truly start to take shape.

As always, this first stages of developing a new version of this project are technical and don’t produce much eye-candy.  However, it is important to keep the interest of the community and the wonderfully functional free prototype SMARCADE: Anarchy Edition is the perfect tool for this job.

The prototype received a minor update last week to adapt to some Netflix changes and I will be announcing a new Screenshot Contest this weekend with some interesting prizes that the community should really enjoy.  Remember that the Steam version will not have a playable BETA until September, so in the mean time SMARCADE: Anarchy Edition is still being fully supported and I am still here to answer any questions that you have about it.

Keep an eye out for the upcoming Screenshot Contest, and as always I would love to hear your feedback; whether positive or negative.  Anarchy Arcade is something I am creating for the community, so it only makes sense that I make it in a way that you’ll enjoy!  Your feedback helps me do this.

Gamer, Designer, Programmer