_____ _____ ____ _____ _____ _____ __ __ _____ _____ _____ _____ | | | \ | | | | _ | | | | | __| | | __| __| | --| | | | | | | --| | | |__| |__| __| | | | | | __| |_____|_|_|_|____/ |_____|__|__|__|__|_____|_____|_____|_|___|_____|_____| ~ about ~
  1. Open sourcing all of cmdchallenge.com

    Thu 11 May 2017

    CMD Challenge is a web application that presents small shell challenges, where submissions are executed remotely in a Docker container. I am happy to announce today that I have decided to release the entire source code and terraform configuration for it under the MIT license on github.

    What this means is that anyone can host their own version of the site easily in the AWS free tier using the exact same configuration that is used for the main production. Kick the tires, clone and go nuts if you want or take a look if you want a very simple example of using API Gateway, Lambda, Dynamo and Docker. The biggest reward for this fun little side-project would be if someone tries to spin up their very own, please let me know on twitter if you do!

    Part of the reason for doing this was to explore how I could simplify the setup using terraform. It is now extremely easy to set up and tear down an isolated and separate version of the site. The only part missing from the automated setup is hosting static assets. The production cmdchallenge.com site uses S3 with cloudfront in front for caching. If you try this out yourself you will need to serve assets locally.

    Before your run the below steps it is important to know what you will be creating, the following AWS resources will be created with terraform:

    • API Gateway
    • Lambda function
    • DynamoDB
    • t2.micro EC2 Instance running coreos

    Here is what you will need to do to create your own cmdchallenge.com in the AWS free tier:

    Step 1: Clone the cmdchallenge-site repository

    git clone https://github.com/jarv/cmdchallenge-site
    git submodule update --init --recursive

    Step 2: Create SSH keys for the EC2 instance

    $ ./bin/create-ssh-keys
    Creating keypair for ssh
    Generating public/private rsa key pair.
    Your identification has been saved in cmd_rsa.
    Your public key has been saved in cmd_rsa.pub.

    This will create a private/ssh directory in the repostory root which contains the private and public keypair for the instance used for cmdchallenge. This private/ directory is ignored by git and should be kept safe. In addition to the ssh keys it will contain the CA and certificates needed for tls authentication for docker after the terraform run.

    Step 3: Update the AWS configuration for terraform

    Modify terraform/site.tf with your AWS credentials. By default it looks for a profile named "cmdchallenge". See the terraform documentation if you want to use something other than an aws profile

    provider "aws" {
      region = "us-east-1"
      shared_credentials_file = "${pathexpand("~/.aws/credentials")}"
      profile = "cmdchallenge"

    Step 4: Run terraform

    cd terraform
    terraform init
    terraform apply

    The process of bringing up all the resources in AWS takes around 4 or 5 minutes. At the very end you will see some terraform outputs:

    ami_id = ami-ad593cbb
    ec2_public_ip =
    invoke_url = https://9hz0doczmb.execute-api.us-east-1.amazonaws.com/prod
    test_hello_world = curl 'https://9hz0doczmb.execute-api.us-east-1.amazonaws.com/prod/?cmd=echo+hello+world&challenge_slug=hello_world'

    Paste the curl command into your terminal to confirm that everything is working:

    $ curl 'https://9hz0doczmb.execute-api.us-east-1.amazonaws.com/prod/?cmd=echo+hello+world&challenge_slug=hello_world'
    {"challenge_slug": "hello_world", "rand_error": false, "output": "hello world", "test_errors": null, "return_code": 0, "correct": true}

    Step 5: Serve assets

    make serve

    Point your browser to http://localhost:8000/ and profit!

  2. Adding checks against random data for shell submissions

    Sun 30 April 2017

    One of the nice features of coding challenge sites like hackerrank is that when you submit an algorithm it will also be checked against random data. This validates the performance of the algorithm and will test it against different inputs. CMD Challenge is a web application that executes shell commands remotely in a Docker container. When it was initially launched, all the checker did was validate the output of the execution against a list of strings, expressed in YAML.

    For example, the "hello world" challenge:

      - slug: hello_world
        description: |
          Print "hello world".
          Hint: There are many ways to print text on
          the command line, one way is with the 'echo'
            - 'hello world'

    When the user submits "echo hello world" the command is executed remotely and the response is validated against the string "hello world". This worked fine for commands that only needed to be verified by checking their output, but what about commands that modify files on disk? For these types of challenges there are a a set of shell scripts that also run inside the container and will return non-zero if any of pre-determined tests fail. This is how the "delete files", "remove files with extension" challenges and other similar ones work to ensure the bash one-line command you entered is correct.

    This solves most of the validation but what about this challenge?

      - slug: sum_all_numbers
        description: |
          The file sum-me.txt has a list of numbers,
          one per line. Print the sum of these numbers.
            - "42"

    One way you can "trick" the site is to simply enter the command "echo 42" which is the correct sum of the numbers in "sum-me.txt". In fact, a lot of random internet people did exactly that as you can see from a small subset of submissions that were previously marked as "correct":

    echo 42
    echo '42'
    echo "42"
    echo "4""2"
    expr 41 + 1
    echo $(( 29 + 13))
    etc. etc.

    So the next natural thing to do is to add a way to randomize the data that the command is operating on. Just like scripts added to run tests, now there are bash scripts for every challenge that allow you to manipulate files before running the command again. Only if the output is still correct (and tests pass, if they are present) will then the challenge be marked as correct.

    For example, for the "sum all numbers" challenge the script looks like this:

    echo $(( $RANDOM % 1000 )) >> sum-me.txt
    paste -sd+ sum-me.txt | bc
    exit 0

    Now if you type the command "echo 42" the randomizer check will fail since will also verify that the command can run against the modified "sum-me.txt" file. Currently, there are a randomizer scripts added for a subset of challenges. If you want to contribute and add more please feel free to send a pull request. If you would like to read some more about how the site is built in an AWS free tier account see the earlier post on building cmdchallenge. If you liked this write up let me know by following @thecmdchallenge on twitter or drop me mail at info@cmdchallenge.com.


  3. Building cmdchallenge using Lambda and API Gateway in the AWS free-tier with Docker and Go

    Mon 24 April 2017

    Have you ever thought about building a side-project for fun without spending a lot on hosting? This post might be for you. With the most tech-buzz-wordy title I could conjure up here is a quick overview of how cmdchallenge.com is built. The site is a simple web application side-project that executes shell commands remotely in a docker container in AWS. The front-end gives the feeling of a normal terminal but underneath it is sending whatever commands you give it remotely on an EC2 instance inside a Docker container.

    The source code for most of it is located on github including a tiny command executer written in Go, the challenge definitions, and a test harness.

    The following AWS services are used for the site:

    • Cloudfront
    • API Gateway
    • S3 bucket
    • Lambda function
    • DynamoDB
    • t2.micro EC2 Instance running coreos
    • CloudWatch logs

    In addition to this Amazon Certificate Manager and Route53 was used but for everything above you can keep costs close to zero in AWS. There is no free tier for Route53 (sad panda) but it's like 50 cents a month for a single zone.


    • Submit commands, execute them in a bash sub-shell.
    • Check the output of the command for different challenges.
    • Run tests for challenges that need them in addition or in place of checking output.

    Deployment tools (simple and boring):

    • Makefiles.
    • Python fabric for running commands and copying files over ssh.
    • Kappa, zips up code, sends it to lambda, also manages Lambda permissions.

    With these tools the following automated steps are taken to deploy the site:

    • Create a Docker image that holds the challenges.
    • Launch a new coreos EC2 instance.
    • Run a fabric script that does the following on the instance over SSH:
      • Configures TLS so that a Lambda function can communicate to Docker on an EC2 instance.
      • Executes some periodic commands to ensure that the host cleans up old containers.
      • Downloads the docker image that has the challenges.
      • Copies the read-only volume that is used on the container for the tests and command runner.
    • Update Lambda with new code.
    • Sync the static assets to S3.
    • Invalidates CF cache for the main site.

    Architecture diagram:

    There are two public entry points for the site, one is the main web-site which is static and served S3. The other is the API gateway at api.cmdchallenge.com which is also fronted by CloudFront so that it can use a certificate from ACM and cache requests.

      api.cmdchallenge.com         cmdchallenge.com
      ********************         ****************
    +---------------------+    +---------------------+ 
    |      Cloudfront     |    |      Cloudfront     |   
    +---------------------+    +---------------------+  
               |                          | 
    +---------------------+         +-----------+
    |    API Gateway      |         | s3 bucket |
    +---------------------+         +-----------+
      |                 |
      | Lambda Function |    +----------+
      |                 |--- |          |
      +-----------------+   \| DynamoDB |
               |             |          |
       +--------------+      +----------+
       | EC2 t2.micro |
       |   (coreos)   |

    One nice thing about using AWS server-less components was that a single t2.micro instance ended up being fine for handling all of the load, even at peak.
    See section on caching/performance below.

    Here is what happens when a command is submitted in the cmdchallenge.com:

    • Javascript code sends an HTTP GET to https://api.cmdchallenge.com
    • If it is cached it returns a response immediately. If not, it forwards the request to the API gateway which in turn sends it to a Lambda function.
    • The Lambda function looks up the challenge and the command in DynamoDb and if it already has an answer it returns that. If the challenge doesn't exist in DyamoDB it is forwarded to the EC2 instance as a command using the docker API.
    • The command that the user provides is passed to a Go command runner that executes the command in a bash sub-shell inside a docker container, checks the output and runs the tests.
    • Results are returned to the Lambda function, it writes them to DynamoDb and returns the response.

    The challenges are expressed in a single YAML, here an example of one challenge:

      - slug: hello_world
        version: 4
        author: cmdchallenge
        description: |
          Print "hello world".
          Hint: There are many ways to print text on
          the command line, one way is with the 'echo'
          Try it below and good luck!
        example: echo 'hello world'
            - 'hello world'

    Interested in coming up with your own? You can submit your own challenge with a pull request. Your challenge will be added to the user-contributed section of the site.


    You may notice that when you do echo hello world on the hello world challenge it returns almost immediately. As it is shown above there are two layers of cache, one at CloudFront and one at DynamoDb to reduce the number of command executions on the Docker container. API Gateway can provide caching but it costs money, I worked around this by sticking CloudFront in front of it but this is only possible with HTTP GETs. With Cloudfront in front the cache-control header in the response from Lambda is set to a very long cache lifetime with every request. The version of the challenge as well as a global cache buster param is passed in so we never have to worry about returning a response from a stale challenge.


    If you are wondering how well this would scale for a lot of traffic, the Lambda function currently dispatches commands to a random host in a statically configured list of EC2 instances making it pretty easy to add more capacity. So far it seems to be operating fine with a single t2.micro EC2 instance handling all command requests that are not cached.

    • Time to get a echo hello world response from a cached cloudfront command - ~50ms
    • Time to get a echo hello world response from a cached command in dynamoDB - ~2.5s
    • Time to get a echo hello world response, executed in a container - ~4s

    Without caching this wouldn't be possible and also the caching at CloudFront enables most commands to return fairly quickly.

    Wrapping up

    If you like the site please follow @thecmdchallenge on twitter or if you have suggestions drop me a mail at info@cmdchallenge.com.


  4. User Submitted Solutions

    Sat 04 March 2017

    Adding to the interesting 191 ways to echo hello world I've now added the ability to see user-submitted solutions to cmdchallenge.

    There are some gems if you dig through them including maybe the longest regex I've ever seen for pulling an IP address out of a file:


    Also scrolling down the page of solutions to the corrupted text problem is glorious.

    The solutions are not updated regularly right now but would be easy enough to do in the future if people want to see more, let me know on twitter and also Update: Solutions are now generated every five minutes. Feel free to submit suggestions for new challenges on github.

  5. figlet breakout

    Fri 24 February 2017

    I was looking for a cool ending for cmdchallenge and decided to dust off a 2 year old javascript project which created a breakout game from figlet fonts. Not quite a full re-write but fixed a lot of bugs and did away completely with coffee-script. More info on the github page.

    Or you can click here to play.

Page 1 / 2 »