hyperpb-go: Parse Protobuf Messages Faster in Go

Overview

hyperpb-go is a Go library for parsing Protobuf messages when message types are dynamic or only known at runtime. It provides a reflection-based alternative to Go's dynamicpb, using a compiled parser tailored to each message type.

It is designed for read-only workloads where parsing speed matters, such as generic services handling schemas from a registry. Unlike generated Protobuf code, it requires runtime compilation of message types and currently does not support mutation of parsed messages.

Key Features

• Compiles parsers from message descriptors or descriptor sets.
• Integrates with Go's Protobuf reflection and proto.Unmarshal APIs.
• Supports reading fields through reflection, including iteration over populated fields.
• Works with reflection-based operations such as JSON marshaling and validation.
• Offers optional profile-guided optimization based on representative messages.
• Provides shared memory reuse for reducing allocation overhead.
• Supports configurable unmarshal behavior and extension resolution.

Use Cases

• Build schema-registry services that receive descriptors and parse messages without generated code.
• Transcode runtime-loaded Protobuf messages to JSON using standard Protobuf APIs.
• Process read-only messages in generic gateways or data pipelines where parsing throughput is important.
• Tune parsing for a known message workload by collecting profiles from representative or sampled traffic.

Project Facts

• Language: Go
• License: Apache-2.0
• Stars: 686
• Forks: 17
• Topics: grpc, protobuf, protoc, protocol-buffers
• Archived: No

Getting Started

Install the Go module:

go get buf.build/go/hyperpb

Compile a message type, create a message, then parse with proto.Unmarshal. See the README for complete examples and options.

Considerations

• The project is experimental, and its API may change substantially before version 1.
• Parsed messages are read-only. Operations that mutate them panic.
• Supported targets are limited to 64-bit amd64 and arm64; other architectures are unsupported.
• Compile and cache message types at runtime. Compilation is intended to be reused, not repeated for every message.
• The reported performance advantage depends on the workload. Benchmark your own message types and parsing patterns before adopting it.