##// END OF EJS Templates
implab » gradle-common
RSS
Rework variant artifacts materialization model...
Rework variant artifacts materialization model Refactor VariantArtifactsPlugin around a live outgoing artifacts context and split artifact publication into explicit internal services: outgoing variant registry, assembly binding, materialization policy hooks, primary-slot convention, and slot assembly handling. Introduce variant artifact slots as identity-first public API and expose materialized assembly handles through ArtifactAssemblies. Add replayable configuration hooks for outgoing configurations, outgoing slots, outgoing variants, and registered assemblies. Create consumable outgoing configurations per variant, bind the primary slot to the root outgoing artifact set, and publish non-primary slots as Gradle outgoing configuration variants. Add deterministic injective task names for slot assembly tasks, use Sync for directory assembly, and configure the default assembly output location under build/variant-assemblies. Make primary-slot selection finalize-on-read and provide a single-slot convention that fails when no unique default can be inferred. Mark artifact internal implementation package as non-public API.
cin - - Load All Authors

File last commit:

r47:6084dc61f02a default
r51:9db7822cd26c default
Show More
variant_sources.md
872 lines | 19.0 KiB | text/x-minidsrc | MarkdownLexer
/ variant_sources.md
History | Source | Raw |Copy content |Copy permalink
cin
WIP working on outgoing variant artifacts
 r47 # Variants and Variant Sources
## Overview
This document describes a two-layer model for build variants:
- `variants` defines the **core domain model**
- `variantSources` defines **source materialization semantics** for that model
The main goal is to keep the core model small, explicit, and stable, while allowing source-related behavior to remain flexible and adapter-friendly.
The model is intentionally split into:
1. a **closed, finalized domain model**
2. an **open, runtime-oriented source materialization model**
This separation is important because compilation, source aggregation, publication, and adapter-specific behavior do not belong to the same abstraction layer.
---
## Core idea
The `variants` model is based on three independent domains:
- `Layer`
- `Role`
- `Variant`
A finalized `VariantsView` contains the normalized relation:
- `(variant, role, layer)`
This relation is the source of truth.
Everything else is derived from it.
---
## `variants`: the core domain model
### Purpose
`variants` describes:
- what layers exist
- what roles exist
- what variants exist
- which `(variant, role, layer)` combinations are valid
It does **not** describe:
- source directories
- source roots
- source set materialization
- compilation tasks
- publication mechanics
- source set inheritance
- layer merge behavior for a concrete toolchain
Those concerns are intentionally outside the core model.
---
## Core DSL example
```groovy
variants {
layers {
main()
test()
generated()
rjs()
cjs()
}
roles {
production()
test()
tool()
}
variant("browser") {
role("production") {
layers("main", "generated", "rjs")
}
role("test") {
layers("main", "test", "generated", "rjs")
}
}
variant("nodejs") {
role("production") {
layers("main", "generated", "cjs")
}
role("test") {
layers("main", "test", "generated", "cjs")
}
role("tool") {
layers("main", "generated", "cjs")
}
}
}
```
### Interpretation
This example means:
* `browser` production uses `main`, `generated`, `rjs`
* `browser` test uses `main`, `test`, `generated`, `rjs`
* `nodejs` production uses `main`, `generated`, `cjs`
* `nodejs` test uses `main`, `test`, `generated`, `cjs`
* `nodejs` tool uses `main`, `generated`, `cjs`
The model is purely declarative.
---
## Identity and references
`Layer`, `Role`, and `Variant` are identity objects.
They exist as declared domain values.
References between model elements are symbolic:
* layers are referenced by layer name
* roles are referenced by role name
* variants are referenced by variant name
This is intentional.
The core model is declarative, not navigation-oriented.
It is acceptable for aggregates to hold symbolic references to foreign domain values, as long as identity is clearly defined and validated later.
---
## Finalization
The `variants` model is finalized once.
Finalization is an internal lifecycle transition. It is typically triggered privately, for example from `afterEvaluate`, but that mechanism is not part of the public API contract.
The public contract is:
* `variants.whenFinalized(...)`
This callback is **replayable**:
* if called before finalization, the action is queued
* if called after finalization, the action is invoked immediately
The callback receives a finalized, read-only view of the model.
Example:
```java
variants.whenFinalized(view -> {
// use finalized VariantsView here
});
```
---
## `VariantsView`
`VariantsView` is the finalized representation of the core model.
It contains:
* all declared `Layer`
* all declared `Role`
* all declared `Variant`
* all normalized entries `(variant, role, layer)`
Conceptually:
```java
interface VariantsView {
Set<Layer> getLayers();
Set<Role> getRoles();
Set<Variant> getVariants();
Set<VariantRoleLayer> getEntries();
}
```
Where:
```java
record VariantRoleLayer(Variant variant, Role role, Layer layer) {}
```
This view is:
* immutable
* normalized
* validated
* independent from DSL internals
---
## Derived views
Two important views can be derived from `VariantsView`:
* `CompileUnitsView`
* `RoleProjectionsView`
These views are not part of the raw core model itself, but they are naturally derived from it.
---
## `CompileUnitsView`
### Purpose
A compile unit is defined as:
* `(variant, layer)`
This is based on the following rationale:
* `variant` defines compilation semantics
* `layer` partitions a variant into separate compilation units
* `role` is not a compilation boundary
This is especially useful for toolchains such as TypeScript, where compilation is often more practical or more correct per layer than for the whole variant at once.
### Example
From:
* `(browser, production, main)`
* `(browser, production, rjs)`
* `(browser, test, main)`
* `(browser, test, test)`
* `(browser, test, rjs)`
we derive compile units:
* `(browser, main)`
* `(browser, rjs)`
* `(browser, test)`
### Conceptual API
```java
interface CompileUnitsView {
Set<CompileUnit> getUnits();
Set<CompileUnit> getUnitsForVariant(Variant variant);
boolean contains(Variant variant, Layer layer);
Set<Role> getRoles(CompileUnit unit);
}
record CompileUnit(Variant variant, Layer layer) {}
```
### Meaning
`CompileUnitsView` answers:
* what can be compiled
* how a variant is partitioned into compile units
* which logical roles include a given compile unit
---
## `RoleProjectionsView`
### Purpose
A role projection is defined as:
* `(variant, role)`
This is based on the following rationale:
* `role` is not about compilation
* `role` groups compile units by purpose
* roles are more closely related to publication, aggregation, assembly, or result grouping
### Example
For `browser`:
* `production` includes compile units:
* `(browser, main)`
* `(browser, rjs)`
* `test` includes compile units:
* `(browser, main)`
* `(browser, test)`
* `(browser, rjs)`
### Conceptual API
```java
interface RoleProjectionsView {
Set<RoleProjection> getProjections();
Set<RoleProjection> getProjectionsForVariant(Variant variant);
Set<RoleProjection> getProjectionsForRole(Role role);
Set<CompileUnit> getUnits(RoleProjection projection);
}
record RoleProjection(Variant variant, Role role) {}
```
### Meaning
`RoleProjectionsView` answers:
* how compile units are grouped by purpose
* what belongs to `production`, `test`, `tool`, etc.
* what should be aggregated or published together
---
## Why `CompileUnitsView` and `RoleProjectionsView` are not part of `VariantsView`
`VariantsView` is intentionally minimal.
It expresses the domain relation:
* `(variant, role, layer)`
`CompileUnitsView` and `RoleProjectionsView` are **derived interpretations** of that relation.
They are natural and useful, but they are still interpretations:
* `CompileUnit = (variant, layer)`
* `RoleProjection = (variant, role)`
This is why they are better treated as derived views rather than direct core model primitives.
---
## `variantSources`: source semantics for layers
### Purpose
`variantSources` does **not** define variants.
It defines how a declared `Layer` contributes sources.
In other words:
* `variants` defines **what exists**
* `variantSources` defines **how layers become source inputs**
This distinction is important.
`variantSources` does not own the variant model. It interprets it.
---
## Main idea
A layer source rule describes the source contribution of a layer.
This is independent of any concrete variant or role.
Conceptually:
* `Layer -> source contribution rule`
Examples of source contribution semantics:
* base directory
* source directories
* logical source kinds (`ts`, `js`, `resources`)
* declared outputs (`js`, `dts`, `resources`)
---
## `variantSources` DSL example
```groovy
variantSources {
layerRule("main") {
from("src/main")
set("ts") {
srcDir("ts")
}
set("js") {
srcDir("js")
}
set("resources") {
srcDir("resources")
}
outputs("js", "dts", "resources")
}
layerRule("test") {
from("src/test")
set("ts") {
srcDir("ts")
}
set("resources") {
srcDir("resources")
}
outputs("js", "dts", "resources")
}
layerRule("rjs") {
from("src/rjs")
set("ts") {
srcDir("ts")
}
outputs("js", "dts")
}
layerRule("cjs") {
from("src/cjs")
set("ts") {
srcDir("ts")
}
outputs("js", "dts")
}
}
```
### Interpretation
This means:
* `main` contributes `ts`, `js`, and `resources`
* `test` contributes `ts` and `resources`
* `rjs` contributes `ts`
* `cjs` contributes `ts`
These are layer rules only.
They do not yet say which variant consumes them.
---
## Why `variantSources` remains open
Unlike `variants`, `variantSources` does not need to be closed in the same way.
Reasons:
* the DSL is internal to source materialization
* the source of truth for unit existence is already finalized in `VariantsView`
* `SourceSetMaterializer` returns `NamedDomainObjectProvider<GenericSourceSet>`
* adapters may need to refine source-related behavior after `variants` is finalized
Therefore:
* `variants` is finalized
* `variantSources` may remain open
This is not a contradiction.
It reflects the difference between:
* a closed domain model
* an open infrastructure/materialization model
This openness is still constrained by explicit policy fixation points:
* late-configuration policy is fixed when the first selector rule is registered
* naming policy is fixed when the finalized `VariantSourcesContext` is created
---
## Late configuration policy
Openness of `variantSources` does not mean that late configuration is
semantically neutral.
Selector rules may be added after the finalized context becomes available, but
their behavior against already materialized `GenericSourceSet` objects must be
controlled explicitly.
Conceptually, `variantSources` exposes a policy choice such as:
```groovy
variantSources {
lateConfigurationPolicy {
failOnLateConfiguration()
}
}
```
Available modes are:
* `failOnLateConfiguration()`
* `warnOnLateConfiguration()`
* `allowLateConfiguration()`
Meaning:
* `fail` rejects selector rules that target already materialized source sets
* `warn` allows them but emits a warning
* `allow` allows them silently
This policy is intentionally modeled as an imperative choice, not as a mutable
property:
* it must be chosen before the first selector rule is added
* selector rules here mean `variant(...)`, `layer(...)`, and `unit(...)`
* once chosen, it cannot be changed later
* it controls runtime behavior, not just a stored value
* the enforcement point is the first selector registration itself, not variants
finalization in isolation
For source sets configured before materialization, selector precedence remains:
```text
variant < layer < unit
```
For already materialized source sets in `warn` and `allow` modes:
* the late action is applied as an imperative follow-up step
* selector precedence is not reconstructed retroactively
* actual observation order is the order in which late actions are registered
---
## Compile-unit naming policy
Source-set naming is treated as a separate policy concern from selector
registration.
Conceptually, `variantSources` exposes:
```groovy
variantSources {
namingPolicy {
failOnNameCollision()
}
}
```
The base projected name of a compile unit is:
```text
variantName + capitalize(layerName)
```
Examples:
* `(browser, main)` -> `browserMain`
* `(browser, rjs)` -> `browserRjs`
Available modes are:
* `failOnNameCollision()` - reject finalized compile-unit models that project
the same source-set name for different compile units
* `resolveNameCollision()` - resolve such conflicts deterministically
### `resolveNameCollision()` semantics
Conflicting compile units are ordered canonically by:
```text
(variant.name, layer.name)
```
Within one conflicting group:
* the first compile unit keeps the base name
* the second gets suffix `2`
* the third gets suffix `3`
* and so on
For example, if:
* `(foo, variantBar)` projects to `fooVariantBar`
* `(fooVariant, bar)` also projects to `fooVariantBar`
then canonical ordering yields:
* `(foo, variantBar)` -> `fooVariantBar`
* `(fooVariant, bar)` -> `fooVariantBar2`
### Fixation point
Naming policy is fixed when the finalized `VariantSourcesContext` is created.
Operationally this means:
* naming policy must be selected before `variantSources.whenFinalized(...)`
becomes observable
* compile-unit names are projected and validated before queued
`whenFinalized(...)` callbacks are replayed
* changing naming policy from inside a `whenFinalized(...)` callback is too late
This differs intentionally from late-configuration policy:
* late-configuration policy is fixed by the first selector rule
* naming policy is fixed by finalized-context creation
---
## `VariantSourcesContext`
`variantSources.whenFinalized(...)` remains useful, but not because `variantSources` itself is frozen.
Its purpose is to provide access to a finalized context derived from `variants`.
This context contains:
* `CompileUnitsView`
* `RoleProjectionsView`
* `SourceSetMaterializer`
By the time the context becomes observable:
* compile-unit naming policy is already fixed
* symbolic source-set names for finalized compile units are already determined
Conceptually:
```java
interface VariantSourcesContext {
CompileUnitsView getCompileUnits();
RoleProjectionsView getRoleProjections();
SourceSetMaterializer getSourceSets();
}
```
This callback is also replayable.
Example:
```java
variantSources.whenFinalized(ctx -> {
var units = ctx.getCompileUnits();
var roles = ctx.getRoleProjections();
var sourceSets = ctx.getSourceSets();
});
```
---
## `SourceSetMaterializer`
### Purpose
`SourceSetMaterializer` is the official source of truth for materialized source sets.
It is responsible for:
* lazy creation of `GenericSourceSet`
* projecting finalized compile units to symbolic source-set names
* validating or resolving name collisions according to naming policy
* applying `layerRule`
* connecting a compile unit to a source set provider
* exposing source sets to adapters
This is the correct place to apply `layerRule`.
Adapters should not apply layer rules themselves.
### Conceptual API
```java
interface SourceSetMaterializer {
NamedDomainObjectProvider<GenericSourceSet> getSourceSet(CompileUnit unit);
}
```
---
## Why `SourceSetMaterializer` should own `layerRule` application
If adapters applied `layerRule` directly, responsibility would leak across multiple layers:
* one component would know compile units
* another would know source semantics
* another would know how to configure `GenericSourceSet`
This would make the model harder to reason about.
Instead:
* `layerRule` is DSL/spec-level
* `SourceSetMaterializer` is execution/materialization-level
* adapters are consumption-level
This gives a much cleaner separation.
---
## `GenericSourceSet` as materialization target
`GenericSourceSet` is the materialized source aggregation object.
It is a good fit because it can represent:
* multiple logical source sets
* aggregated source directories
* declared outputs
* lazy registration through providers
The materializer is therefore the owner of:
* creating `GenericSourceSet`
* populating its source sets
* declaring outputs
* returning a provider for later use
---
## How an adapter should use the model
Example:
```java
variantSources.whenFinalized(ctx -> {
for (CompileUnit unit : ctx.getCompileUnits().getUnits()) {
var sourceSetProvider = ctx.getSourceSets().getSourceSet(unit);
var variant = unit.variant();
var layer = unit.layer();
// create compile task for this compile unit
// configure compiler options from variant semantics
// use sourceSetProvider as task input
}
for (RoleProjection projection : ctx.getRoleProjections().getProjections()) {
var units = ctx.getRoleProjections().getUnits(projection);
// aggregate outputs of included compile units
// use for publication or assembly
}
});
```
---
## Why compile unit is `(variant, layer)` and not `(variant, role)` or `(variant, role, layer)`
### Not `(variant, role)`
Because role is not a compilation boundary.
Role is a logical grouping of results.
### Not `(variant, role, layer)`
Because role does not define the compile unit itself.
A compile unit is a unit of compilation, not a unit of publication grouping.
### Correct interpretation
* `(variant, layer)` = compile unit
* `(variant, role)` = logical result group
* `(variant, role, layer)` = membership relation between them
This is the most coherent separation.
---
## Model boundaries
### What belongs to `variants`
* declared domains: `Layer`, `Role`, `Variant`
* normalized relation `(variant, role, layer)`
* finalization lifecycle
* finalized `VariantsView`
### What belongs to derived views
* compile units: `(variant, layer)`
* role projections: `(variant, role)`
### What belongs to `variantSources`
* source semantics of layers
* source materialization rules
* lazy `GenericSourceSet` provisioning
* source adapter integration
### What does not belong to `variants`
* source directories
* base paths
* output declarations
* source set layout
* task registration
* compiler-specific assumptions
---
## Design principles
### 1. Keep the core model small
The core model should only contain domain facts.
### 2. Separate domain truth from materialization
The existence of compile units comes from `VariantsView`, not from source rules.
### 3. Treat source materialization as infrastructure
`variantSources` is an interpretation layer, not the source of truth.
### 4. Prefer replayable finalized hooks
Adapters should not depend on raw Gradle lifecycle callbacks such as `afterEvaluate`.
### 5. Make late behavior explicit
Late configuration after materialization is a policy decision, not an implicit
guarantee.
### 6. Keep heavy runtime objects behind providers
Materialized `GenericSourceSet` objects should remain behind a lazy API.
### 7. Make name-collision behavior explicit
Compile-unit naming must be governed by an explicit policy, not by incidental
materialization order.
---
## Summary
The model is intentionally split into two layers.
### `variants`
A closed, finalized domain model:
* `Layer`
* `Role`
* `Variant`
* `(variant, role, layer)`
### `variantSources`
An open, source-materialization layer:
* layer source rules
* compile-unit source set materialization
* compile-unit naming policy
* adapter-facing `GenericSourceSet` providers
### Derived views
From the finalized variant model:
* `CompileUnitsView`: `(variant, layer)`
* `RoleProjectionsView`: `(variant, role)`
### Operational interpretation
* `variant` defines compilation semantics
* `layer` partitions compilation
* `role` groups results by purpose
This keeps the core model stable and minimal, while allowing source handling and adapter integration to remain flexible.