Sixpack: A Language-Agnostic A/B Testing Framework

Introduction

Sixpack is a powerful, language-agnostic A/B testing framework that enables developers to conduct experiments across diverse programming environments. It simplifies the process of A/B testing by exposing a straightforward HTTP API, allowing client libraries in virtually any language to interact with it. The framework comprises two main components: Sixpack-server, which handles web requests and experiment logic, and Sixpack-web, an optional web dashboard for visualizing and managing your A/B tests.

Installation

Getting started with Sixpack is straightforward. First, ensure you have Redis (>= 2.6) and Python (>= 2.7) installed. It's recommended to use a virtual environment.

To install Sixpack, use pip:

$ pip install sixpack

After installation, you'll need to create a configuration file (e.g., config.yml). A basic configuration includes Redis connection details and other settings:

redis_port: 6379
redis_host: localhost
redis_prefix: sixpack
redis_db: 15
secret_key: '<your secret key here>'

Once configured, you can start the Sixpack server:

$ SIXPACK_CONFIG=<path to config.yml> sixpack

The server will listen on port 5000 by default. The optional Sixpack-web dashboard can be started similarly:

$ SIXPACK_CONFIG=<path to config.yml> sixpack-web

The dashboard runs on port 5001 by default and requires a secret_key to be set.

Examples

All interactions with Sixpack are performed via HTTP GET requests, using a unique client_id to link participation to conversion.

Participating in an Experiment

To have a user participate in an experiment, send a GET request to the /participate endpoint:

$ curl http://localhost:5000/participate?experiment=button_color&alternatives=red&alternatives=blue&client_id=12345678-1234-5678-1234-567812345678

Sixpack will automatically create the experiment if it doesn't exist and assign an alternative to the client_id.

Converting a User

Once a user has completed the desired action, record a conversion by sending a GET request to the /convert endpoint:

$ curl http://localhost:5000/convert?experiment=button_color&client_id=12345678-1234-5678-1234-567812345678

Sixpack internally handles which alternative the client_id was assigned to for conversion tracking.

Why Use Sixpack

Sixpack offers several compelling reasons for adoption:

• Language Agnostic: Its HTTP API allows integration with any programming language, making it highly flexible for diverse tech stacks.
• Simple API: Interaction is via straightforward HTTP GET requests, simplifying client-side implementation.
• Built-in Dashboard: Sixpack-web provides a convenient dashboard for real-time tracking, visualization, and management of experiments, including selecting winners.
• Deterministic Algorithm: Utilizing an algorithm ported from Facebook's Planout project, Sixpack ensures consistent and reliable alternative assignment.
• Extensive Client Libraries: Official and community-contributed client libraries are available for popular languages like Ruby, Python, JavaScript, PHP, iOS, Go, Perl, C#, and Java/Android, accelerating integration.

Links

• GitHub Repository: https://github.com/sixpack/sixpack
• Client Libraries:
  • Ruby: http://github.com/seatgeek/sixpack-rb
  • Python: http://github.com/seatgeek/sixpack-py
  • JavaScript: http://github.com/seatgeek/sixpack-js
  • PHP: http://github.com/seatgeek/sixpack-php
  • iOS: http://github.com/seatgeek/sixpack-ios
  • Go: http://github.com/subosito/sixpack-go
  • Perl: http://github.com/b10m/p5-WWW-Sixpack
  • C#: https://github.com/nderraugh/sixpack-cs
  • Java/Android: http://github.com/seatgeek/sixpack-java
• License: BSD 2-Clause License