Acre Design & Architecture

Acre is an web hosting platform that runs user-contributed javascript code in a secure fashion, using the Mozilla Rhino Javascript interpreter.

Security is achieved by sandboxing the user-contributed code twice: first at the Javascript interpreter level then at the Java Virtual Machine level, plus other isolation barriers added by the operating system and network layers. This software virtualization allows Acre to host a large number of applications on a single machine.

Moreover, the complete statelessness of Acre applications allows us to easily scale the system horizontally on many separate servers to meet whatever traffic needs and to achieve improved fault tolerance.

Overview

The Acre platform is composed of two main parts:

  1. Acre App Editor: the client side web application that talks via AJAX calls to the HTTP web services provided by Acre and Freebase and serves as an integrated development environment

  2. Acre Application Server: the HTTP server responsible of securely executing user-contributed code

Here is a simplified overview of the system:

Acre Architecture Diagram

Designing for Security and Isolation

When dealing with user-contributed code, security and isolation are critical, not only isolation between the user code and the hosting environment (because that code is in fact running behind our firewalls), but also between the applications themselves and Freebase.com to prevent cross-side scripting (XSS) attacks.

This influenced Acre's design on many level:

  • The need to avoid XSS issues forced us to host each individual application on a separate virtual host, but behind the scenes there are just a handful of Acre processes on a handful of machines handling a potentially very large number of virtual hosts.

  • The Acre javascript API provided to the user-contributed code is the only way the code can interact with the system. Just like on a web browser, this API was carefully designed to avoid security-sensitive operations (like read/writing from the local disk).

  • Two levels of sandboxing on virtualized environments greatly reduce the risk of buffer-overflow attacks from user-code

Data Storage

Almost all the data that Acre uses is stored directly in Freebase, including all the user scripts, the application metadata, the virtual host to app resolution information, the user identifiers and the access control rules.

Bytecode Compilation of Scripts and Templates

While the scripts source code is stored in Freebase, Acre does not need to fetch it every time they are requested, but only the first time they change. When unchanged, Acre uses Rhino's internals to compile javascript code into java bytecode and stores the resulting classes in memory (in special isolated classloaders). This on-line bytecode compilation allows very fast execution and allows the JIT compiler to further optimize performance of the compiled scripts over time when they don't change.

Acre templates are converted internally into Javascript code first and then follow the exact same path that other scripts take. This has the advantage of reducing complexity and potential surface for external attacks because templates being ultimately scripts means that nothing more can be done in a template than in a script.

Resource-Limited Threads

One of the most difficult part of Acre's development had to do with protection from starvation of server resources: a lot of thought and effort went into defining ways in which Java threads could be limited in both the amount of time they could run and the amount of memory they could consume. One first tier of protection in that sense is offered directly by Rhino, which is capable of stopping a script after a certain number of calls or instructions. Unfortunately we found this not to be enough as standard Javascript APIs offer functionality such as regular expression evaluation that is provided directly by the JVM and could be used by malicious code to bypass protections at the javascript level. Moreover, Rhino didn't offer any protection against memory allocation abuse.

Unfortunately, the standard Java Virtual Machines does not either. This forced us to develop a modified Java virtual machine that contains 'resource-limited' threads (limited in both execution time and memory space). This very low level resource consumption protection, coupled with Acre's tracking of scripts calling other scripts by using the same quota limits, allows Acre to run scripts without worry of them maliciously starving resources.

Note that this is not a prevention against Denial of Service attacks (which should happen higher up in the HTTP tier chain), but a prevention against resource starvation coming directly from Acre apps.