##// END OF EJS Templates
implab » gradle-common
RSS
WIP working on outgoing variant artifacts
WIP working on outgoing variant artifacts
cin - - Load All Authors

File last commit:

r47:6084dc61f02a default
r47:6084dc61f02a default
Show More
variant_sources.md
872 lines | 19.0 KiB | text/x-minidsrc | MarkdownLexer
/ variant_sources.md
History | Source | Raw |Copy content |Copy permalink
cin
WIP working on outgoing variant artifacts
 r47 # Variants and Variant Sources
## Overview
This document describes a two-layer model for build variants:
- `variants` defines the **core domain model**
- `variantSources` defines **source materialization semantics** for that model
The main goal is to keep the core model small, explicit, and stable, while allowing source-related behavior to remain flexible and adapter-friendly.
The model is intentionally split into:
1. a **closed, finalized domain model**
2. an **open, runtime-oriented source materialization model**
This separation is important because compilation, source aggregation, publication, and adapter-specific behavior do not belong to the same abstraction layer.
---
## Core idea
The `variants` model is based on three independent domains:
- `Layer`
- `Role`
- `Variant`
A finalized `VariantsView` contains the normalized relation:
- `(variant, role, layer)`
This relation is the source of truth.
Everything else is derived from it.
---
## `variants`: the core domain model
### Purpose
`variants` describes:
- what layers exist
- what roles exist
- what variants exist
- which `(variant, role, layer)` combinations are valid
It does **not** describe:
- source directories
- source roots
- source set materialization
- compilation tasks
- publication mechanics
- source set inheritance
- layer merge behavior for a concrete toolchain
Those concerns are intentionally outside the core model.
---
## Core DSL example
```groovy
variants {
layers {
main()
test()
generated()
rjs()
cjs()
}
roles {
production()
test()
tool()
}
variant("browser") {
role("production") {
layers("main", "generated", "rjs")
}
role("test") {
layers("main", "test", "generated", "rjs")
}
}
variant("nodejs") {
role("production") {
layers("main", "generated", "cjs")
}
role("test") {
layers("main", "test", "generated", "cjs")
}
role("tool") {
layers("main", "generated", "cjs")
}
}
}
```
### Interpretation
This example means:
* `browser` production uses `main`, `generated`, `rjs`
* `browser` test uses `main`, `test`, `generated`, `rjs`
* `nodejs` production uses `main`, `generated`, `cjs`
* `nodejs` test uses `main`, `test`, `generated`, `cjs`
* `nodejs` tool uses `main`, `generated`, `cjs`
The model is purely declarative.
---
## Identity and references
`Layer`, `Role`, and `Variant` are identity objects.
They exist as declared domain values.
References between model elements are symbolic:
* layers are referenced by layer name
* roles are referenced by role name
* variants are referenced by variant name
This is intentional.
The core model is declarative, not navigation-oriented.
It is acceptable for aggregates to hold symbolic references to foreign domain values, as long as identity is clearly defined and validated later.
---
## Finalization
The `variants` model is finalized once.
Finalization is an internal lifecycle transition. It is typically triggered privately, for example from `afterEvaluate`, but that mechanism is not part of the public API contract.
The public contract is:
* `variants.whenFinalized(...)`
This callback is **replayable**:
* if called before finalization, the action is queued
* if called after finalization, the action is invoked immediately
The callback receives a finalized, read-only view of the model.
Example:
```java
variants.whenFinalized(view -> {
// use finalized VariantsView here
});
```
---
## `VariantsView`
`VariantsView` is the finalized representation of the core model.
It contains:
* all declared `Layer`
* all declared `Role`
* all declared `Variant`
* all normalized entries `(variant, role, layer)`
Conceptually:
```java
interface VariantsView {
Set<Layer> getLayers();
Set<Role> getRoles();
Set<Variant> getVariants();
Set<VariantRoleLayer> getEntries();
}
```
Where:
```java
record VariantRoleLayer(Variant variant, Role role, Layer layer) {}
```
This view is:
* immutable
* normalized
* validated
* independent from DSL internals
---
## Derived views
Two important views can be derived from `VariantsView`:
* `CompileUnitsView`
* `RoleProjectionsView`
These views are not part of the raw core model itself, but they are naturally derived from it.
---
## `CompileUnitsView`
### Purpose
A compile unit is defined as:
* `(variant, layer)`
This is based on the following rationale:
* `variant` defines compilation semantics
* `layer` partitions a variant into separate compilation units
* `role` is not a compilation boundary
This is especially useful for toolchains such as TypeScript, where compilation is often more practical or more correct per layer than for the whole variant at once.
### Example
From:
* `(browser, production, main)`
* `(browser, production, rjs)`
* `(browser, test, main)`
* `(browser, test, test)`
* `(browser, test, rjs)`
we derive compile units:
* `(browser, main)`
* `(browser, rjs)`
* `(browser, test)`
### Conceptual API
```java
interface CompileUnitsView {
Set<CompileUnit> getUnits();
Set<CompileUnit> getUnitsForVariant(Variant variant);
boolean contains(Variant variant, Layer layer);
Set<Role> getRoles(CompileUnit unit);
}
record CompileUnit(Variant variant, Layer layer) {}
```
### Meaning
`CompileUnitsView` answers:
* what can be compiled
* how a variant is partitioned into compile units
* which logical roles include a given compile unit
---
## `RoleProjectionsView`
### Purpose
A role projection is defined as:
* `(variant, role)`
This is based on the following rationale:
* `role` is not about compilation
* `role` groups compile units by purpose
* roles are more closely related to publication, aggregation, assembly, or result grouping
### Example
For `browser`:
* `production` includes compile units:
* `(browser, main)`
* `(browser, rjs)`
* `test` includes compile units:
* `(browser, main)`
* `(browser, test)`
* `(browser, rjs)`
### Conceptual API
```java
interface RoleProjectionsView {
Set<RoleProjection> getProjections();
Set<RoleProjection> getProjectionsForVariant(Variant variant);
Set<RoleProjection> getProjectionsForRole(Role role);
Set<CompileUnit> getUnits(RoleProjection projection);
}
record RoleProjection(Variant variant, Role role) {}
```
### Meaning
`RoleProjectionsView` answers:
* how compile units are grouped by purpose
* what belongs to `production`, `test`, `tool`, etc.
* what should be aggregated or published together
---
## Why `CompileUnitsView` and `RoleProjectionsView` are not part of `VariantsView`
`VariantsView` is intentionally minimal.
It expresses the domain relation:
* `(variant, role, layer)`
`CompileUnitsView` and `RoleProjectionsView` are **derived interpretations** of that relation.
They are natural and useful, but they are still interpretations:
* `CompileUnit = (variant, layer)`
* `RoleProjection = (variant, role)`
This is why they are better treated as derived views rather than direct core model primitives.
---
## `variantSources`: source semantics for layers
### Purpose
`variantSources` does **not** define variants.
It defines how a declared `Layer` contributes sources.
In other words:
* `variants` defines **what exists**
* `variantSources` defines **how layers become source inputs**
This distinction is important.
`variantSources` does not own the variant model. It interprets it.
---
## Main idea
A layer source rule describes the source contribution of a layer.
This is independent of any concrete variant or role.
Conceptually:
* `Layer -> source contribution rule`
Examples of source contribution semantics:
* base directory
* source directories
* logical source kinds (`ts`, `js`, `resources`)
* declared outputs (`js`, `dts`, `resources`)
---
## `variantSources` DSL example
```groovy
variantSources {
layerRule("main") {
from("src/main")
set("ts") {
srcDir("ts")
}
set("js") {
srcDir("js")
}
set("resources") {
srcDir("resources")
}
outputs("js", "dts", "resources")
}
layerRule("test") {
from("src/test")
set("ts") {
srcDir("ts")
}
set("resources") {
srcDir("resources")
}
outputs("js", "dts", "resources")
}
layerRule("rjs") {
from("src/rjs")
set("ts") {
srcDir("ts")
}
outputs("js", "dts")
}
layerRule("cjs") {
from("src/cjs")
set("ts") {
srcDir("ts")
}
outputs("js", "dts")
}
}
```
### Interpretation
This means:
* `main` contributes `ts`, `js`, and `resources`
* `test` contributes `ts` and `resources`
* `rjs` contributes `ts`
* `cjs` contributes `ts`
These are layer rules only.
They do not yet say which variant consumes them.
---
## Why `variantSources` remains open
Unlike `variants`, `variantSources` does not need to be closed in the same way.
Reasons:
* the DSL is internal to source materialization
* the source of truth for unit existence is already finalized in `VariantsView`
* `SourceSetMaterializer` returns `NamedDomainObjectProvider<GenericSourceSet>`
* adapters may need to refine source-related behavior after `variants` is finalized
Therefore:
* `variants` is finalized
* `variantSources` may remain open
This is not a contradiction.
It reflects the difference between:
* a closed domain model
* an open infrastructure/materialization model
This openness is still constrained by explicit policy fixation points:
* late-configuration policy is fixed when the first selector rule is registered
* naming policy is fixed when the finalized `VariantSourcesContext` is created
---
## Late configuration policy
Openness of `variantSources` does not mean that late configuration is
semantically neutral.
Selector rules may be added after the finalized context becomes available, but
their behavior against already materialized `GenericSourceSet` objects must be
controlled explicitly.
Conceptually, `variantSources` exposes a policy choice such as:
```groovy
variantSources {
lateConfigurationPolicy {
failOnLateConfiguration()
}
}
```
Available modes are:
* `failOnLateConfiguration()`
* `warnOnLateConfiguration()`
* `allowLateConfiguration()`
Meaning:
* `fail` rejects selector rules that target already materialized source sets
* `warn` allows them but emits a warning
* `allow` allows them silently
This policy is intentionally modeled as an imperative choice, not as a mutable
property:
* it must be chosen before the first selector rule is added
* selector rules here mean `variant(...)`, `layer(...)`, and `unit(...)`
* once chosen, it cannot be changed later
* it controls runtime behavior, not just a stored value
* the enforcement point is the first selector registration itself, not variants
finalization in isolation
For source sets configured before materialization, selector precedence remains:
```text
variant < layer < unit
```
For already materialized source sets in `warn` and `allow` modes:
* the late action is applied as an imperative follow-up step
* selector precedence is not reconstructed retroactively
* actual observation order is the order in which late actions are registered
---
## Compile-unit naming policy
Source-set naming is treated as a separate policy concern from selector
registration.
Conceptually, `variantSources` exposes:
```groovy
variantSources {
namingPolicy {
failOnNameCollision()
}
}
```
The base projected name of a compile unit is:
```text
variantName + capitalize(layerName)
```
Examples:
* `(browser, main)` -> `browserMain`
* `(browser, rjs)` -> `browserRjs`
Available modes are:
* `failOnNameCollision()` - reject finalized compile-unit models that project
the same source-set name for different compile units
* `resolveNameCollision()` - resolve such conflicts deterministically
### `resolveNameCollision()` semantics
Conflicting compile units are ordered canonically by:
```text
(variant.name, layer.name)
```
Within one conflicting group:
* the first compile unit keeps the base name
* the second gets suffix `2`
* the third gets suffix `3`
* and so on
For example, if:
* `(foo, variantBar)` projects to `fooVariantBar`
* `(fooVariant, bar)` also projects to `fooVariantBar`
then canonical ordering yields:
* `(foo, variantBar)` -> `fooVariantBar`
* `(fooVariant, bar)` -> `fooVariantBar2`
### Fixation point
Naming policy is fixed when the finalized `VariantSourcesContext` is created.
Operationally this means:
* naming policy must be selected before `variantSources.whenFinalized(...)`
becomes observable
* compile-unit names are projected and validated before queued
`whenFinalized(...)` callbacks are replayed
* changing naming policy from inside a `whenFinalized(...)` callback is too late
This differs intentionally from late-configuration policy:
* late-configuration policy is fixed by the first selector rule
* naming policy is fixed by finalized-context creation
---
## `VariantSourcesContext`
`variantSources.whenFinalized(...)` remains useful, but not because `variantSources` itself is frozen.
Its purpose is to provide access to a finalized context derived from `variants`.
This context contains:
* `CompileUnitsView`
* `RoleProjectionsView`
* `SourceSetMaterializer`
By the time the context becomes observable:
* compile-unit naming policy is already fixed
* symbolic source-set names for finalized compile units are already determined
Conceptually:
```java
interface VariantSourcesContext {
CompileUnitsView getCompileUnits();
RoleProjectionsView getRoleProjections();
SourceSetMaterializer getSourceSets();
}
```
This callback is also replayable.
Example:
```java
variantSources.whenFinalized(ctx -> {
var units = ctx.getCompileUnits();
var roles = ctx.getRoleProjections();
var sourceSets = ctx.getSourceSets();
});
```
---
## `SourceSetMaterializer`
### Purpose
`SourceSetMaterializer` is the official source of truth for materialized source sets.
It is responsible for:
* lazy creation of `GenericSourceSet`
* projecting finalized compile units to symbolic source-set names
* validating or resolving name collisions according to naming policy
* applying `layerRule`
* connecting a compile unit to a source set provider
* exposing source sets to adapters
This is the correct place to apply `layerRule`.
Adapters should not apply layer rules themselves.
### Conceptual API
```java
interface SourceSetMaterializer {
NamedDomainObjectProvider<GenericSourceSet> getSourceSet(CompileUnit unit);
}
```
---
## Why `SourceSetMaterializer` should own `layerRule` application
If adapters applied `layerRule` directly, responsibility would leak across multiple layers:
* one component would know compile units
* another would know source semantics
* another would know how to configure `GenericSourceSet`
This would make the model harder to reason about.
Instead:
* `layerRule` is DSL/spec-level
* `SourceSetMaterializer` is execution/materialization-level
* adapters are consumption-level
This gives a much cleaner separation.
---
## `GenericSourceSet` as materialization target
`GenericSourceSet` is the materialized source aggregation object.
It is a good fit because it can represent:
* multiple logical source sets
* aggregated source directories
* declared outputs
* lazy registration through providers
The materializer is therefore the owner of:
* creating `GenericSourceSet`
* populating its source sets
* declaring outputs
* returning a provider for later use
---
## How an adapter should use the model
Example:
```java
variantSources.whenFinalized(ctx -> {
for (CompileUnit unit : ctx.getCompileUnits().getUnits()) {
var sourceSetProvider = ctx.getSourceSets().getSourceSet(unit);
var variant = unit.variant();
var layer = unit.layer();
// create compile task for this compile unit
// configure compiler options from variant semantics
// use sourceSetProvider as task input
}
for (RoleProjection projection : ctx.getRoleProjections().getProjections()) {
var units = ctx.getRoleProjections().getUnits(projection);
// aggregate outputs of included compile units
// use for publication or assembly
}
});
```
---
## Why compile unit is `(variant, layer)` and not `(variant, role)` or `(variant, role, layer)`
### Not `(variant, role)`
Because role is not a compilation boundary.
Role is a logical grouping of results.
### Not `(variant, role, layer)`
Because role does not define the compile unit itself.
A compile unit is a unit of compilation, not a unit of publication grouping.
### Correct interpretation
* `(variant, layer)` = compile unit
* `(variant, role)` = logical result group
* `(variant, role, layer)` = membership relation between them
This is the most coherent separation.
---
## Model boundaries
### What belongs to `variants`
* declared domains: `Layer`, `Role`, `Variant`
* normalized relation `(variant, role, layer)`
* finalization lifecycle
* finalized `VariantsView`
### What belongs to derived views
* compile units: `(variant, layer)`
* role projections: `(variant, role)`
### What belongs to `variantSources`
* source semantics of layers
* source materialization rules
* lazy `GenericSourceSet` provisioning
* source adapter integration
### What does not belong to `variants`
* source directories
* base paths
* output declarations
* source set layout
* task registration
* compiler-specific assumptions
---
## Design principles
### 1. Keep the core model small
The core model should only contain domain facts.
### 2. Separate domain truth from materialization
The existence of compile units comes from `VariantsView`, not from source rules.
### 3. Treat source materialization as infrastructure
`variantSources` is an interpretation layer, not the source of truth.
### 4. Prefer replayable finalized hooks
Adapters should not depend on raw Gradle lifecycle callbacks such as `afterEvaluate`.
### 5. Make late behavior explicit
Late configuration after materialization is a policy decision, not an implicit
guarantee.
### 6. Keep heavy runtime objects behind providers
Materialized `GenericSourceSet` objects should remain behind a lazy API.
### 7. Make name-collision behavior explicit
Compile-unit naming must be governed by an explicit policy, not by incidental
materialization order.
---
## Summary
The model is intentionally split into two layers.
### `variants`
A closed, finalized domain model:
* `Layer`
* `Role`
* `Variant`
* `(variant, role, layer)`
### `variantSources`
An open, source-materialization layer:
* layer source rules
* compile-unit source set materialization
* compile-unit naming policy
* adapter-facing `GenericSourceSet` providers
### Derived views
From the finalized variant model:
* `CompileUnitsView`: `(variant, layer)`
* `RoleProjectionsView`: `(variant, role)`
### Operational interpretation
* `variant` defines compilation semantics
* `layer` partitions compilation
* `role` groups results by purpose
This keeps the core model stable and minimal, while allowing source handling and adapter integration to remain flexible.