The security of machine learning algorithms has not yet received enough attention from the public, an attacker might intentionally manipulate the input data of machine learning algorithms to compromise the system security. An example of this is shown in the following picture, after adding some intentionally designed random noise into the original image, the machine learning model has 99% confidence in predicting the image as a "gibbon", even though the change in the image is neglectable to human eyes.

Since machine learning has become critical parts of many systems in different fields, such as autonomous driving, we created AdvEx to prevent threat like this. AdvEx is a web service for assessing the robustness of machine learning models with adversarial machine learning. It is designed to fulfill the following quality attributes:

• Scalability: achieved by auto-scaling and load-balancing using elastic beanstalk
• Availability: achieved by having different servers in different availability zones
• Performance: achieved by using GPU in evaluation workers, and having users upload their models directly to S3 buckets without going through servers first
• Security: achieved by using AWS security group
• Usability: achieved by creating helpful instructions and tutorials for users who are not machine learning experts
• Configurability: achieved by using Elastic Beanstalk, Docker and config files

Links:

Project video demo | Front-end static demo | Front-end repo | Back-end repo | Evaluation worker repo

• Supports RESTful APIs for user and submission data, implemented with Flask and PostgreSQL
• Adopts an automatic build, test and deploy workflow using Docker, pytest and Travis-CI, improves code quality with continuous integration
• Handles auto-scaling and load-balancing with Elastic Beanstalk and Docker, reduces deployment time of a new version to 5 min

python version 3.6

• Flask==1.0.2
• Flask-Session==0.3.1
• Flask-CORS==3.0.6
• Flask-SQLAlchemy==2.3.2
• Sqlalchemy-Utils==0.33.3
• Numpy==1.14.2
• Pytest==3.6.0
• Psycopg2==2.7.5
• boto3==1.7
• pytest-flask==0.10.0

Docker available at docker hub. For pulling docker:

docker pull awp135/advex:backend

Supports two ways of deploying:

First, set configuration settings in environment variables, an example of this is (see examples of configuration in docs/ for detail):

export SQLALCHEMY_DATABASE_URI="your_postgressql_uri"
export SQLALCHEMY_TRACK_MODIFICATIONS=False
export DEBUG=True
export HOST='0.0.0.0'
export PORT=5000

Then, start the backend server by simply:

./start.sh

First, set the environment variables on AWS just as testing locally. Then, start the backend server by (the following command maps host port 4000 to docker port 80 and uses 5432 for postgresDB port, see examples of configuration in docs/ for detail):

docker run -p 4000:80 awp135/advex:backend