Repository navigation
CBEF-128: Generate integer keys from a Couchbase sequence by default - #189
davidkelly wants to merge 1 commit into
Conversation
f5777cc to
1e20133
Compare
1e20133 to
4eda206
Compare
emilienbev
left a comment
There was a problem hiding this comment.
Really like this one. Assigning sequence values at save time keeps Add free of I/O, and CouchbaseSequenceLocation puts the location logic in one place. A few small things, and one question about the batch query.
|
|
||
| // NEXT VALUE FOR increments once per document, so one value per array element. The array | ||
| // comes back as one JSON string because ExecuteScalar returns a single value. | ||
| var query = $"SELECT RAW ENCODE_JSON(ARRAY NEXT VALUE FOR {sequence} FOR x IN ARRAY_RANGE(0, {count}) END)"; |
There was a problem hiding this comment.
This gets all the values from one row, so it depends on NEXT VALUE FOR running once per array element. The comment says it increments once per document, and this query has a single document. Is the per-element behaviour documented for every server version we support? If not, maybe SELECT RAW NEXT VALUE FOR s FROM ARRAY_RANGE(0, n) AS x with a reader is safer, and it drops the ENCODE_JSON step.
| /// EF Core set from one) is in the update entry only: EF Core copies it to the CLR object when | ||
| /// it accepts the changes, after the documents are written. | ||
| /// </remarks> | ||
| internal static string GetLegacyPrimaryKey( |
There was a problem hiding this comment.
Small thing. This copies GetPrimaryKey(IEntityType, object) and only changes where the values come from, and the old method has no caller in src now. Could we keep one loop that takes a value reader?
| /// </summary> | ||
| /// <param name="dependencies">The selector's dependencies.</param> | ||
| /// <param name="optionsBuilder">Unused; kept so existing service registrations still resolve.</param> | ||
| public CouchbaseValueGeneratorSelector( |
There was a problem hiding this comment.
Small thing. Nothing creates CouchbaseSequenceValueGenerator anymore. Only its IsTypeSupported is still used. Could we move that check to a small helper and remove the generator, or mark it [Obsolete] if we want to keep the public API? The unused optionsBuilder parameter could go too, since DI resolves the constructor without it.
| // successful insert/update regardless of whether it's also a concurrency token (harmless and | ||
| // keeps the property useful for inspection even without concurrency enforcement). | ||
| /// <summary> | ||
| /// Gives every added entry whose key uses a Couchbase sequence its value from the sequence, as |
There was a problem hiding this comment.
Nit. This method landed between the [CouchbaseMeta(Cas)] comment and GetCasProperty, so that comment now sits on the wrong method. Could we move the new method above it?
| /// </summary> | ||
| private async Task GenerateSequenceValuesAsync(IList<IUpdateEntry> entries, CancellationToken cancellationToken) | ||
| { | ||
| Dictionary<(CouchbaseSequenceLocation, CouchbaseSequenceOptions), List<(IUpdateEntry, IProperty)>>? groups = null; |
There was a problem hiding this comment.
Nit. The options only feed the error message, but they're part of the group key. Two properties on the same sequence with different options would cause two queries. Could we group by location only?
| public class ConferencePlannerCouchbaseTest(ConferencePlannerCouchbaseTest.ConferencePlannerCouchbaseFixture fixture) | ||
| : ConferencePlannerTestBase<ConferencePlannerCouchbaseTest.ConferencePlannerCouchbaseFixture>(fixture) | ||
| { | ||
| // The base tests rely on a rolled-back transaction to undo their changes; Couchbase does not |
There was a problem hiding this comment.
FYI, this works around a rollback that doesn't undo changes. That sounds like a provider issue rather than a harness one. Could it get a row in root-causes.tsv, so it stays visible until someone looks at it?
| int-key-not-generated Insert failed for key 0 in keyspace .*\n.*DocumentExistsException A generated integer key gets no value, so every new entity is written as document 0 | ||
| seed-doc-exists Insert failed for key .*\n.*DocumentExistsException Seeding inserts a key that already exists (owned types with explicit keys, among others) | ||
| seed-empty-key Insert failed for key in keyspace Document key is empty: the save path reads the key by CLR reflection, not from the update entry | ||
| key-getter Property Get method was not found Document key read by CLR reflection fails for a property with no getter |
There was a problem hiding this comment.
FYI, the save path now reads key values from the update entry, so the key-getter row might be fixed too. Worth checking in the full CI run before merge?
Couchbase has no identity columns, and nothing generated a value for an
integer key that EF Core marks as generated on add. EF kept a temporary
value in the change tracker, the save path read 0 from the entity, and
every new entity was written as document "0": the second insert failed
with DocumentExistsException.
- CouchbaseKeySequenceConvention gives such a key (single property,
integer, generated on add, no explicit generator) a sequence named
<EntityName>Sequence. Derived types share the root's sequence.
- Values are assigned at SaveChanges: Add gets a temporary value from
the relational selector, with no I/O, so a synchronous Add works.
The database wrapper fetches all the values each sequence needs in
one query (ENCODE_JSON(ARRAY NEXT VALUE FOR ... ARRAY_RANGE(0, N)))
and sets them as store-generated values, like an identity column:
EF updates the dependents' foreign keys, and the entity gets the key
when the save completes. A failed save keeps the temporary values.
- The legacy document key reads its values from the update entry, in
the old part order, so a composite key that includes a generated
foreign key is not written as "0_0".
- A missing sequence (query error 19106) throws InvalidOperationException
with the CREATE SEQUENCE statement to run.
- TPC: CouchbaseValueGenerationConvention lets TPC integer keys be
generated (the relational convention does not, as an identity column
is per table), and the model validator accepts a sequence-backed TPC
key. Any other generated TPC key still gets EF's warning.
- CouchbaseModelValidator was never registered. It is now, with its
empty ValidatePropertyMapping override removed: that override would
have disabled EF's check for unsupported property types.
- CouchbaseSequenceLocation is now the one place that resolves a
sequence's bucket and scope, for value generation, EnsureCreated and
EnsureDeleted. The scope is still the context's, so existing
UseSequence sequences are not moved.
Unit tests cover the convention, TPC validation, temporary values on
Add and sequence locations; integration tests cover a synchronous Add,
foreign keys, a key the application sets, a missing sequence, a
read-only-before-save key, and a composite key with a
sequence-generated foreign key.
Docs (sequences.md) and CHANGELOG describe the default and the
collision risk with existing integer-keyed documents.
Spec harness: CouchbaseSpecModel.Apply(generateIntKeys: true) emulated
this fix in 28 fixtures and also forced TPC keys that EF configures as
ValueGeneratedNever (GearsOfWar's Faction.Id) to be generated, which
failed every TPC GearsOfWar test with TpcStoreGeneratedIdentityWarning.
The emulation is gone; the harness only starts test sequences at
1,000,000, above the explicit keys EF's fixtures seed.
ConferencePlanner reseeds after each test: SessionsController_Put now
passes and its rename leaked into Search("C#").
4eda206 to
19e506c
Compare
Couchbase has no identity columns, and nothing generated a value for an integer key that EF
Core marks as generated on add: every new entity was written as document
0, and the secondinsert failed with
DocumentExistsException. This gives such keys a Couchbase sequence bydefault. Stacked on #188 (which is on #190), so the diff here shows only this change.
Change
CouchbaseKeySequenceConvention: a single-property integer key generated on add, with noexplicit generator, uses a sequence named
<EntityName>Sequence. Derived types share theroot's sequence, so TPC keys are unique across collections.
Addgets a temporary value and does no I/O, so it workssynchronously; the SaveChanges interceptor fetches all the values each sequence needs in one
query, and EF updates the dependents' foreign keys.
InvalidOperationExceptionwith theCREATE SEQUENCEstatement torun.
EnsureCreatedAsynccreates the sequences.CouchbaseModelValidatorwas never registered. It is now, without its emptyValidatePropertyMappingoverride, which would have disabled EF's check for unsupportedproperty types.
sequences.md, including the collision risk with existing integer-keyed documents) andCHANGELOG.
configures as
ValueGeneratedNeverto be generated, which is what failed every TPCGearsOfWar test. Test sequences now start at 1,000,000, above the keys EF's fixtures seed.
Deletes the
int-key-not-generatedandtpc-key-generationrows fromroot-causes.tsv.Tests
Add, foreign keys, akey the app sets, a missing sequence).
TPCGearsOfWar 0 → 846/1,183.
known-failures.txtwill be regenerated from a full CI runbefore merge.
Jira: CBEF-128