Апстрим master-1006-0886f11

docs/edit.md

@@ -85,6 +85,9 @@ If presets are insufficient, you can manually configure the following parameters
 | `vlm_max_size` | int | Maximum VLM input size; interpreted according to `vlm_resize_mode`. | Integer |
 | `vlm_min_size` | int | Minimum VLM input size; interpreted according to `vlm_resize_mode`. | Integer |
 | `vlm_size` | int | Shortcut to set both VLM min and max size to the same value. | Integer |
+| `ref_image_size` | string | MiniMax-H3 reference image sizing before VAE encoding. `match` scales each reference down to the generation's pixel area; `max` uses a 2048 px short edge. Only references above the limit are downscaled; each axis is then snapped to the nearest 32 px multiple (an axis of at least 16 px grows by at most 16 px, and references are never narrower than 32 px). | `match`, `max` |
+
+`ref_image_size` applies to `vid_gen` with MiniMax-H3 Ref2VA. Reference latents are attended to at every sampling step, so `max` can be several times slower.
 
 ### Preset Default Values
 

docs/generation_metadata.md

@@ -0,0 +1,165 @@
+# Generation results and metadata
+
+`generate_image()`, `generate_video()`, `adetail_image()` and `upscale()` return
+owned results containing the media and its generation record. This is a breaking
+C API change: applications and bindings must update their calls and rebuild.
+
+```c
+sd_img_gen_params_t params;
+sd_img_gen_params_init(&params);
+params.prompt = "a small house";
+
+sd_image_generation_result_t* result = NULL;
+if (generate_image(ctx, &params, &result)) {
+    for (int i = 0; i < result->count; ++i) {
+        const sd_image_result_t* item = &result->items[i];
+        /* Consume item->image and item->generation_info together. */
+    }
+    free_sd_image_generation_result(result);
+}
+```
+
+`result_out` is required and is set to `NULL` on failure. A successful result
+owns its pixels and UTF-8 JSON strings. The video result additionally owns its
+audio and contains the effective `fps` and `frame_count`. Results remain valid
+after freeing the generation context. Use the matching library result destructor;
+do not separately free fields unless transferring ownership and clearing them.
+Both result destructors accept `NULL`.
+
+## Reproduction parameters
+
+The `sdcpp.generation/v1` JSON is a snapshot of generation parameters for the
+same loaded context and the same input images, masks, reference media and other
+assets. It is created after `GenerationRequest` resolves the request, before
+preprocessing or sampling. A failed call returns no result or metadata.
+
+Fields follow the C generation parameter structs: `prompt`, `negative_prompt`,
+`width`, `height`, `seed`, `sample_params`, `hires`, `cache`,
+`vae_tiling_params`, `image_preprocess`, and the image/video-specific options.
+`kind` distinguishes image and video parameters. There is no inferred `task`.
+Enums use their API string names. Seeds are decimal strings to preserve their
+full integer range. Omitted optional fields retain the parameter initializer's
+defaults, including automatic values represented by infinity in the C API.
+
+Disabled optional features are omitted entirely: `cache`, `hires`, VAE tiling,
+zero/nonpositive `shifted_timestep`, inactive SLG, false circular axes, and empty
+preprocessing/reference arguments. SLG layers remain present when `slg_uncond`
+is enabled even with zero SLG scale. Enabled caches store their mode's parameters;
+TaylorSeer retains the shared DBCache thresholds and limits used by the runtime.
+Image CFG at its neutral value of 1 is omitted and resolves back to 1 by default.
+
+Image strength, ControlNet/IP-Adapter strength and PhotoMaker settings are stored
+when their corresponding input images are supplied. PuLID requires an embedding
+path. Layer count is stored for layered generation; video VACE and high-noise
+settings require those models. High-noise settings omit scheduler, custom sigmas,
+flow shift and timestep shifting because that pass shares the base configuration.
+Explicit nonempty argument strings are retained without rewriting their contents.
+Core sampling values such as seed, sampler, scheduler, steps and guidance remain
+explicit even when equal to their defaults. ADetailer always retains its inpaint
+strength because its input crops are created later.
+Model-required VAE tiling is resolved before serialization, so forced tiling
+(such as MiniMax-H3) remains present even if the caller did not enable it.
+
+The request fixes the seed, dimensions, clip skip, guidance, sampler, scheduler
+and eta before taking the snapshot. `custom_sigmas` is stored only when supplied
+as an input. Automatically computed schedules, preprocessing transforms,
+conditioning/cache decisions, detector regions and output statistics are not
+stored: they are recomputed from the same parameters and inputs.
+
+Each image records the seed used to sample its latent. `batch_count` is omitted;
+replay image parameters with `batch_count = 1`. Layers or frames decoded from
+the same latent share its seed, so layered generation reproduces that group of
+layers. Video frames share the video request. Partial batch cancellation returns
+metadata only for completed outputs. Full cancellation returns no result.
+
+The current image pipeline prepares input latents once using the starting seed
+and shares them across a batch. With a stochastic VAE encoder, replaying a later
+image alone using its sampling seed can change those input latents. Per-image
+seeds therefore do not guarantee independent reproduction for these paths.
+
+To replay a record, initialize the matching C parameter struct, populate its
+fields from the JSON, restore the same input buffers, and call generation with
+the same context. Keep custom-sigma/SLG arrays and string storage alive for the
+call. Paths supplied as generation parameters (LoRAs, hires models and identity
+embeddings) are retained. `models` identifies successfully loaded context model
+files by component (`model`, `diffusion_model`, `clip_l`, `vae`, etc.), using
+filenames with extensions and omitting directories. Optional components disabled
+for the request and preview-only TAE are omitted. WebUI text includes `Model`
+(the diffusion model when supplied separately, otherwise the checkpoint, without
+the extension) and `VAE` for a separate active VAE. These names are identifiers,
+not hashes or sufficient configuration to rebuild a context; replay still requires
+the same model files and context settings. The JSON does not embed media buffers
+or model weights.
+
+When `--lora-model-dir` is configured, CLI/server output records express LoRA
+paths relative to that directory, preserving subdirectories and file extensions
+and using `/` separators. This applies to file metadata, API responses and
+ADetailer records. Resolve these paths against the same LoRA directory before
+passing them back to the C API. Core results retain the supplied loading paths.
+Files outside the directory retain `..` components; paths on another volume,
+or records without a configured LoRA directory, retain their original paths.
+
+## Postprocessing
+
+Upscale records `upscale.upscale_factor`. ADetailer records `adetailer` and
+`inpaint_params`, with a concrete starting seed chosen before detection. These
+operations require the same upscaler/detector context and source image on replay.
+They do not store detection results or per-region generation traces.
+
+The CLI/server append operation parameter snapshots to `postprocess` in call
+order. The source generation parameters remain unchanged. Frame postprocessing
+uses the same operation parameters for each video frame. Standalone
+postprocessing does not invent source diffusion parameters for an input image.
+
+## File metadata
+
+Image encoders accept the native record and derive WebUI infotext from it.
+Images use the existing metadata locations and combined text layout:
+`<WebUI infotext>, SDCPP: <native JSON>`. Standalone postprocessing without
+source diffusion parameters stores only native JSON. Video metadata stores
+only native JSON.
+
+| Container | Metadata location |
+| --- | --- |
+| PNG | `tEXt`, keyword `parameters` |
+| JPEG | COM segment, `parameters` label followed by a NUL byte |
+| WebP image / animated WebP | XMP property `sdcpp:parameters` |
+| AVI | RIFF INFO `ICMT` |
+| WebM | `COMMENT` tag |
+
+Inspect images or videos with `sd-cli -M metadata --image <file>`; the same
+command supports all containers above, with `--metadata-format json` for a
+structured report. Video metadata inspection does not require a model or video
+decoder.
+Matroska containers are also supported, including `.webm` outputs whose audio
+codec causes the muxer to select the Matroska document type.
+
+The XMP namespace is `https://github.com/leejet/stable-diffusion.cpp/ns/1.0/`.
+PNG and JPEG retain the existing UTF-8 text payload convention. Memory encoding
+and file saving use the same implementation. The core result and API
+`generation_info` fields remain pure JSON; the combined layout applies only
+to embedded image metadata.
+
+WebUI conversion is lossy: known samplers and schedules have explicit name
+mappings; unsupported ones use `SDCPP Sampler`/`SDCPP Schedule` fields. Hires
+keeps the base request size and separate configured hires size/steps. The native
+record retains the full generation parameters. Standalone postprocessing has no WebUI
+infotext because its original diffusion settings are unknown.
+
+WebUI prompt text appends ordinary LoRAs as `<lora:filename_without_extension:weight>`.
+Directory components are omitted to match WebUI's filename lookup; the native
+`loras` array retains relative paths and weights. High-noise-only LoRAs and names
+that cannot be represented by this tag syntax remain in the native record only.
+The native prompt is not modified, and postprocessing LoRAs are not added to the
+source generation prompt.
+
+JPEG requires the COM payload, including the label and terminating NUL bytes,
+to fit within 65,533 bytes. Larger records fail encoding with an error rather
+than being truncated. Use PNG or WebP for larger records.
+
+Disabling file metadata omits both image representations, but does not disable
+the core generation record or its API response. Native async image results and
+OpenAI image response items contain `generation_info`; video results contain a
+sequence-level `generation_info`. The SD API `info` object contains per-output
+`generation_info` alongside derived `infotexts` and each image's seed. Its outer
+`parameters` field continues to echo the request.

docs/image_preprocessing.md

@@ -128,7 +128,8 @@ To disable reference resizing before VAE encoding, use
 `--ref-image-args "resize_before_vae=false"` or the server field
 `"ref_image_args": "resize_before_vae=false"`. This is separate from
 `target=ref,mode=none`, which only skips input geometry. Model constraints
-still apply.
+still apply; MiniMax-H3 requires 32-pixel-aligned references and warns instead
+of honoring the switch.
 
 ## Server requests
 
@@ -148,20 +149,23 @@ an array of rule strings:
 OpenAI-compatible requests accept it through
 `<sd_cpp_extra_args>{...}</sd_cpp_extra_args>` in the prompt.
 Request rules replace server-default rules. Generation metadata records the
-user rules; image encodings and channel conventions are unchanged.
+user rules for replay with the same input images; image encodings and channel conventions are unchanged.
 
 ## C API
 
 Set `image_preprocess` on the existing image/video generation parameters.
-The `generate_image()` and `generate_video()` signatures are unchanged:
+Generation returns media and actual parameters together; see [generation metadata](generation_metadata.md):
 
 ```c
 sd_img_gen_params_t params;
 sd_img_gen_params_init(&params);
 /* Set prompt, original-resolution input images, and generation options. */
 params.image_preprocess.rules = "target=init,mode=crop-resize,filter=lanczos;"
                                 "target=mask,filter=nearest-exact";
-bool ok = generate_image(ctx, &params, &images, &count);
+sd_image_generation_result_t* result = NULL;
+bool ok = generate_image(ctx, &params, &result);
+/* Consume result->items and their generation_info on success. */
+free_sd_image_generation_result(result);
 ```
 
 Both generation parameter initializers set `image_preprocess.rules` to `NULL`,

docs/minimax_h3.md

@@ -80,8 +80,20 @@ latents condition the diffusion transformer. Paired video and audio references
 share the same timeline. Ref2VA cannot be combined with `--init-img` or
 `--end-img` in one request.
 
-Reference images keep their aspect ratio and are only downscaled when their
-pixel area exceeds the requested generation canvas.
+Reference images keep their aspect ratio and are only downscaled when they
+exceed the configured limit; neither mode scales a reference up. The limit is
+selected by `ref_image_size`. The default, `match`, uses the pixel area of the
+requested generation canvas. Pass `--ref-image-args "ref_image_size=max"` (or
+`ref_image_size` in `sd_vid_gen_params_t.ref_image_args`) to use a 2048 px short
+edge instead, which preserves more identity detail at a higher sampling cost:
+reference latents are attended at every sampling step.
+
+Whichever limit applies, each axis is then snapped to the nearest 32-pixel
+multiple. An axis of at least 16 pixels can therefore grow by up to 16 pixels
+past the scaled size (a 48-pixel axis becomes 64), and references are never
+narrower than 32 pixels.
+`resize_before_vae=false` is not supported here; MiniMax-H3 keeps references on
+the 32-pixel grid and reports a warning.
 
 The C API exposes the same inputs through `ref_images`, `ref_videos`, and
 `ref_audios` in `sd_vid_gen_params_t`. Each `sd_ref_video_t` supplies its own