Schema: Data Generation

Konfigo's schema allows you to define generators that create new data within your configuration. Generators construct values from existing configuration data and variables, enabling dynamic configuration composition.

Available Generators

Konfigo provides several built-in generators:

• concat: Combines configuration data, variables, and literal text
• timestamp: Generates current timestamp in various formats
• random: Generates random values (integers, floats, strings, UUIDs)
• id: Generates various types of identifiers using alphanumeric characters

concat Generator

The concat generator creates string values by combining configuration data, variable substitutions, and literal text. It operates in two phases: first replacing {placeholder} tokens with values from configuration paths, then resolving ${VARIABLE} references.

Structure

generators:
  - type: "concat"
    targetPath: "path.to.new.key"
    format: "template with {placeholders} and ${VARIABLES}"
    sources:
      placeholder_name: "config.path.to.value"

Fields

• type (Required): Must be "concat"
• targetPath (Required): Dot-separated path where the generated value will be placed
• format (Required): Template string with {placeholders} and ${VARIABLES}
• sources (Required): Map of placeholder names to configuration paths

Processing Order

1. Replace {placeholder} tokens using sources mapping
2. Resolve ${VARIABLE} references from vars and environment
3. Insert result at targetPath

timestamp Generator

The timestamp generator creates timestamp values in various formats.

Structure

generators:
  - type: "timestamp"
    targetPath: "path.to.timestamp"
    format: "rfc3339"  # Optional, defaults to "rfc3339"

Fields

• type (Required): Must be "timestamp"
• targetPath (Required): Dot-separated path where the timestamp will be placed
• format (Optional): Timestamp format (defaults to "rfc3339")

Supported Formats

• unix: Unix timestamp (seconds since epoch) - e.g., 1640995200
• unixmilli: Unix timestamp in milliseconds - e.g., 1640995200000
• rfc3339: RFC3339 format - e.g., 2021-12-31T16:00:00Z
• iso8601: ISO8601 format - e.g., 2021-12-31T16:00:00Z
• Custom: Any Go time format string - e.g., 2006-01-02 15:04:05

Example

generators:
  - type: "timestamp"
    targetPath: "metadata.createdAt"
    format: "rfc3339"
  - type: "timestamp"
    targetPath: "metadata.buildTime"
    format: "2006-01-02 15:04:05"

random Generator

The random generator creates random values in various formats.

Structure

generators:
  - type: "random"
    targetPath: "path.to.random.value"
    format: "string:16"  # Required

Fields

• type (Required): Must be "random"
• targetPath (Required): Dot-separated path where the random value will be placed
• format (Required): Random value format specification

Supported Formats

• int:min:max: Random integer between min and max (inclusive) - e.g., int:1:100
• float:min:max: Random float between min and max - e.g., float:0.0:1.0
• string:length: Random string using [a-zA-Z0-9] - e.g., string:16
• bytes:length: Random bytes as hex string - e.g., bytes:8
• uuid: UUID v4 format - e.g., uuid

Examples

generators:
  - type: "random"
    targetPath: "service.port"
    format: "int:8000:9000"
  - type: "random"
    targetPath: "session.id"
    format: "string:32"
  - type: "random"
    targetPath: "request.uuid"
    format: "uuid"

id Generator

The id generator creates various types of identifiers using alphanumeric characters.

Structure

generators:
  - type: "id"
    targetPath: "path.to.id"
    format: "simple:8"  # Optional, defaults to "simple:8"

Fields

• type (Required): Must be "id"
• targetPath (Required): Dot-separated path where the ID will be placed
• format (Optional): ID format specification (defaults to "simple:8")

Supported Formats

• simple:length: Random ID using [a-zA-Z0-9] - e.g., simple:12
• prefix:prefix:length: ID with prefix + random chars - e.g., prefix:user_:8
• numeric:length: Numeric ID using [0-9] - e.g., numeric:6
• alpha:length: Alphabetic ID using [a-zA-Z] - e.g., alpha:10
• sequential: Sequential counter-based ID (starts from 1)
• timestamp: Timestamp + random suffix - e.g., 1640995200abcd

Examples

generators:
  - type: "id"
    targetPath: "user.id"
    format: "prefix:usr_:8"     # Results in: usr_A9Kx2mP1
  - type: "id"
    targetPath: "session.counter"
    format: "sequential"        # Results in: 1, 2, 3, ...
  - type: "id"
    targetPath: "trace.id"
    format: "timestamp"         # Results in: 1640995200A9Kx

Examples from Tests

Basic Generation

Input Configuration:

service:
  name: "data-processor"
  instanceId: "instance-007"
  port: 8080
region: "us-west-2"

Schema:

vars:
  - name: "APP_VERSION"
    value: "1.2.3"
generators:
  - type: "concat"
    targetPath: "service.identifier"
    format: "Service: {name} (ID: {id}) running in {region}"
    sources:
      name: "service.name"
      id: "service.instanceId"
      region: "region"

Result:

service:
  identifier: "Service: data-processor (ID: instance-007) running in us-west-2"
  # ... other fields preserved

Multiple Generators with Variables

Schema:

vars:
  - name: "APP_VERSION"
    value: "1.2.3"
  - name: "DOMAIN"
    value: "example.com"
generators:
  - type: "concat"
    targetPath: "service.url"
    format: "https://{service}.${DOMAIN}:{port}"
    sources:
      service: "service.name"
      port: "service.port"
  - type: "concat"
    targetPath: "database.connectionString"
    format: "postgresql://{host}:{port}/{db}"
    sources:
      host: "database.host"
      port: "database.port"
      db: "database.name"
  - type: "concat"
    targetPath: "service.fullIdentifier"
    format: "{name}-{version} - ${APP_VERSION} ({env})"
    sources:
      name: "service.name"
      version: "service.version"
      env: "environment"

Result:

service:
  url: "https://data-processor.example.com:8080"
  fullIdentifier: "data-processor-1.2.3 - 1.2.3 (production)"
database:
  connectionString: "postgresql://db-server:5432/app_db"

Variables-Only Generation

Generators can use only variables without configuration sources:

Schema:

vars:
  - name: "APP_VERSION"
    value: "1.2.3"
  - name: "DOMAIN"
    value: "example.com"
generators:
  - type: "concat"
    targetPath: "metadata.buildInfo"
    format: "Built version ${APP_VERSION} for ${DOMAIN}"
    sources: {}  # No configuration sources needed

Error Handling

Common errors and their solutions:

Missing Source Path

# ERROR: 'service.missing' not found in configuration
sources:
  name: "service.missing"  # This path doesn't exist

Empty Format

# ERROR: format cannot be empty
format: ""

No Sources When Needed

# ERROR: Missing sources for placeholder {name}
format: "Service: {name}"
sources: {}  # Missing required source mapping

Advanced Patterns

Cascading Generation

Generators can reference previously generated values:

generators:
  - type: "concat"
    targetPath: "temp.namespace"
    format: "{app}-{env}"
    sources:
      app: "metadata.appName"
      env: "environment"
  - type: "concat"
    targetPath: "kubernetes.namespace"
    format: "k8s-{namespace}"
    sources:
      namespace: "temp.namespace"  # Reference previous generation

Deep Path Creation

Generators automatically create nested paths:

generators:
  - type: "concat"
    targetPath: "deeply.nested.generated.value"
    format: "Generated at {path}"
    sources:
      path: "metadata.location"

Creates the entire path structure if it doesn't exist.

Best Practices

1. Clear Naming: Use descriptive placeholder names in sources
2. Path Validation: Ensure source paths exist in your configuration
3. Variable Order: Define variables before generators that use them
4. Avoid Conflicts: Don't overwrite critical configuration paths
5. Test Generation: Verify generated values match expectations

• sources (Required, map): A map where:
  • Keys are the placeholder_name strings used in the format string (without the curly braces).
  • Values are dot-separated paths to existing values within the current state of the configuration (after merges, but typically before most other schema processing for the current generator pass).

How it Works:

1. For each placeholder defined in sources: a. Konfigo retrieves the value from the configuration at the specified path. b. If a source path is not found, Konfigo will return an error.
2. The format string is processed: a. Each {placeholder_name} is replaced with the corresponding value retrieved from sources.
3. The resulting string (after sources substitution) is then processed for standard Konfigo variable substitution (e.g., ${MY_VARIABLE}).
4. The final, fully resolved string is set at the targetPath in the configuration.

Example:

Schema (schema.yml):

vars:
  - name: "APP_VERSION"
    value: "1.2.3"
config: # Assume this is the state of the config before this generator runs
  service:
    name: "data-processor"
    instanceId: "instance-007"
  region: "us-west-2"

generators:
  - type: "concat"
    targetPath: "service.identifier"
    format: "Service: {name} (ID: {id}) running in {region_val} - Version: ${APP_VERSION}"
    sources:
      name: "service.name"
      id: "service.instanceId"
      region_val: "region"
  - type: "concat"
    targetPath: "service.url"
    format: "https://${FQDN_VAR}" # Using only a global variable
    sources: {} # No local sources needed if format string only uses global vars

Variables (e.g., from -V vars.yml or environment):

# vars.yml
FQDN_VAR: "myapp.example.com"

Processing Steps:

1. First Generator (service.identifier):

   • sources:
     • name -> service.name -> "data-processor"
     • id -> service.instanceId -> "instance-007"
     • region_val -> region -> "us-west-2"
   • format after sources substitution: "Service: data-processor (ID: instance-007) running in us-west-2 - Version: ${APP_VERSION}"
   • format after ${APP_VERSION} substitution: "Service: data-processor (ID: instance-007) running in us-west-2 - Version: 1.2.3"
   • config.service.identifier becomes "Service: data-processor (ID: instance-007) running in us-west-2 - Version: 1.2.3"

2. Second Generator (service.url):

   • sources: (empty)
   • format after sources substitution (no change): "https://${FQDN_VAR}"
   • format after ${FQDN_VAR} substitution: "https://myapp.example.com"
   • config.service.url becomes "https://myapp.example.com"

Resulting Configuration (snippet):

service:
  name: "data-processor"
  instanceId: "instance-007"
  identifier: "Service: data-processor (ID: instance-007) running in us-west-2 - Version: 1.2.3"
  url: "https://myapp.example.com"
region: "us-west-2"
# ... other config ...

Generators are applied in the order they are defined in the generators list. The output of one generator can potentially be used as a source for a subsequent generator if its targetPath is referenced in the later generator's sources.