66 lines
2.5 KiB
Markdown
66 lines
2.5 KiB
Markdown
# System Patterns
|
|
|
|
## 1. Overall Architecture
|
|
- Modular Go application following standard Go project layout (`cmd`, `pkg`, `test`).
|
|
- Configuration-driven behavior using YAML.
|
|
- Clear separation of concerns between packages:
|
|
- `cmd/benchmark`: Main application entry point, CLI flags, orchestrates initialization.
|
|
- `pkg/config`: Loading and validation of `config.yaml`.
|
|
- `pkg/client`: Abstraction layer for HTTP requests.
|
|
- `pkg/tokenizer`: Handles prompt generation and token counting.
|
|
- `pkg/concurrency`: Manages concurrent workers and request scheduling.
|
|
- `pkg/stats`: Collects and aggregates performance metrics.
|
|
- `pkg/report`: (Planned) Generates final benchmark reports.
|
|
|
|
## 2. Key Technical Decisions & Patterns
|
|
- **HTTP Client Abstraction:**
|
|
- `HTTPClient` interface (`pkg/client/interface.go`) defines `Do` (non-streaming) and `Stream` methods.
|
|
- `FastHTTPClient` (`pkg/client/fasthttp.go`) is the primary implementation:
|
|
- Uses `fasthttp` library for high performance in the `Do` method.
|
|
- Uses Go's standard `net/http` library internally for the `Stream` method for better SSE handling stability.
|
|
- Uses `sync.Pool` for `fasthttp` request/response object reuse.
|
|
- **Concurrency Management:**
|
|
- `pkg/concurrency/manager.go` uses a goroutine pool (implemented via channels/semaphore pattern) to limit concurrent workers based on `config.Concurrency`.
|
|
- Uses `context.Context` for managing benchmark duration and worker cancellation.
|
|
- **Statistics Collection:**
|
|
- `pkg/stats/collector.go` uses a `sync.Map` for thread-safe storage of individual `RequestResult` structs.
|
|
- Calculates final aggregate statistics (average, min/max, percentiles) upon benchmark completion.
|
|
- Percentiles calculated using `gonum/stat`.
|
|
- **Token Handling:**
|
|
- Uses `github.com/tiktoken-go/tokenizer` for generating prompts of specific token lengths and counting tokens in responses.
|
|
- **Configuration:**
|
|
- Uses `viper` (or similar YAML parsing library) within `pkg/config` to load settings.
|
|
|
|
## 3. Component Relationships
|
|
```mermaid
|
|
flowchart TD
|
|
A[cmd/benchmark/main.go] --> B(pkg/config)
|
|
A --> C{client.HTTPClient}
|
|
A --> D(pkg/stats)
|
|
A --> E(pkg/concurrency)
|
|
A --> F(pkg/tokenizer)
|
|
|
|
C -->|Do uses| G[fasthttp]
|
|
C -->|Stream uses| H[net/http]
|
|
|
|
E --> C
|
|
E --> F
|
|
E --> D
|
|
|
|
subgraph pkg/client
|
|
C
|
|
G
|
|
H
|
|
end
|
|
|
|
subgraph pkg/stats
|
|
D
|
|
I[gonum/stat]
|
|
end
|
|
|
|
subgraph pkg/tokenizer
|
|
F
|
|
J[tiktoken-go]
|
|
end
|
|
```
|