sixpack: Run A/B Tests Across Multiple Languages

Overview

Sixpack is a server-side A/B-testing framework for teams whose applications use different programming languages. A central HTTP API assigns experiment alternatives consistently to a client ID and records conversions, so separate language-specific applications can participate in the same testing setup.

The project also provides an optional web dashboard for viewing experiment results, choosing winners, and editing descriptions. Its Python server uses Redis for storage. See the repository and project homepage for details.

Key Features

• Exposes experiment participation and conversion endpoints over HTTP GET requests.
• Uses a client ID to link a user's assigned alternative with later conversions.
• Selects alternatives deterministically so a client receives the same choice for an experiment.
• Creates experiments and KPIs automatically when they are first used.
• Supports multiple KPIs, traffic-fraction limits, and optional forced alternatives for testing.
• Offers an optional web dashboard to compare alternatives, select winners, and edit experiment descriptions.
• Includes a health-check endpoint and optional StatsD metrics.
• Lists client libraries for several languages, including Ruby, Python, JavaScript, PHP, iOS, Go, Perl, C#, and Java/Android.

Use Cases

• Product teams can test interface variants across web services written in multiple languages using one shared experiment service.
• Application developers can assign a consistent variant to a user and record a later conversion without implementing experiment storage themselves.
• Analysts and product owners can review experiment outcomes in the optional dashboard and select a winning alternative.
• Teams can gradually expose a test to a fraction of traffic before including more users.

Project Facts

• Language: Python
• License: BSD-2-Clause
• Stars: 1.8k
• Forks: 179
• Topics: none listed
• Archived: No

Getting Started

Install the Python package with pip install sixpack. Redis is required. For server configuration, launch instructions, and API details, see the README.

Considerations

• The server requires Redis 2.6 or later. The README lists Python 2.7 or later and notes that Python 3.0 was untested.
• The dashboard is optional and requires a configured secret key; it runs separately from the API server.
• Experiment names are not validated by the service, and the README warns that unrestricted names can grow the Redis keyspace. Add an appropriate whitelist if experiment creation needs to be constrained.
• The repository was last pushed on 2022-08-21, so assess current maintenance and compatibility before adopting it for a new deployment.