How it works
A core that knows no database decides every fake. Sources speak each upstream's protocol and say which field each value came from.
A query, end to end
- Your agent sends
SELECT * FROM users WHERE email = 'kim@example.net'to Sosie’s local address. - Guard. Sosie parses it: a read, so it is allowed. A write never leaves Sosie.
- Turn back.
kim@example.netis the fake of a real email, so Sosie puts the real email in its place. - Read. The query runs upstream on a read-only session, with a timeout and a row cap.
- Fake. Postgres says which table and column each result column came from. Sosie fakes every value by that field, and the same real value always gets the same fake.
- The agent gets rows of fakes, with the real shape: the same types, formats and relationships.
The faking model
- Fake by default. Every value is faked. A value passes unchanged only when
sosie.ymllists its field. - Consistent. A real value in a field always becomes the same fake. A foreign key fakes like the key it points to,
so
orders.user_idmatchesusers.id, and joins hold. - Realistic. The fake has the field’s type and, where the field’s name says so, a realistic format: emails, names, phone numbers, addresses. The name only picks the format; it never lets a value pass.
- Reversible inside Sosie only. The store maps fakes back to real values, so queries by a fake work. Real values are sealed in the store and never leave Sosie’s process.
- Schema first, detection last. The database’s schema says what each value is. Automatic detection never decides that a value is safe.
Domains: which values share fakes
A domain is the set of real values that share fakes. For a field:
- Listed under
pass: no domain; the value passes. - It holds a MongoDB ObjectId: the one domain every ObjectId shares, so a reference to a document (
orders.userId) is the fake of its_id. - It shares another field’s values (a foreign key, or a view’s column): that field’s domain.
orders.user_idfakes likeusers.id, and a key of two characters stays two characters wherever it appears. - Its format is personal (email, phone, first name, last name, full name, street, city, postal code): the format’s domain. One person’s email is the same fake in every field, and in every source: the database and the API.
- Otherwise: the field’s own.
Values a query computes are faked by their type, in a domain of their own.
Fakes by type
| Type | Fake |
|---|---|
| Text | The field’s format when known. Otherwise shaped like the value: an email stays an email, a date a date, a code like AB-1234 keeps its letters, digits and punctuation, and words become words of about the same length |
| Integer | The same sign and number of digits; never zero for a value that is not zero |
| Decimal | Digits replaced: the same sign, digits and scale |
| UUID | A random UUID |
| ObjectId | 24 hex digits: its time shifted by up to a year, the rest random |
| Boolean | Flipped or not, decided once per field |
| Enum | A label of the same enum, by a fixed shuffle per field |
| Date, time | Shifted by up to a year, the same shift for the same value; a fraction of a second as long, its digits drawn but the last. A month in text (2019-03) by 1 to 12 months |
| Interval | Another duration in the same form, each number of as many digits |
| Bits | As many bits, each drawn, no two values of a field given one fake while it has bits to spare. A single bit is a flag: flipped or not, decided once per field, as a boolean is |
| Range | Each bound faked as its type, then put in order |
| JSON | The same structure; each leaf faked by its JSON type, or by the format its key names. A key that looks like a word (city, createdAt) stays; one that is a value (an email, a name) is faked |
| Array | Each element faked in the field’s domain |
| IP address | An address from a documentation range |
| Bytes | Random bytes of the same length |
A value with nothing to fake ('', -, NaN) stays as it is. Fakes are deterministic: each is drawn from a seed
taken from the real value’s keyed hash, so the same value gives the same fake in every session.
Query by what you saw
A fake compared with a field (email = '…', IN (…), = ANY(…), an API path or query parameter, a GraphQL argument)
is looked up in that field’s domain and replaced with the real value before the query runs. Booleans and enum labels
turn back by their field’s flip and shuffle. A fake Sosie never gave is used as written, so it filters on whatever
real value it happens to be.
Sosie never shows a value it turned back. If the answer holds it, whole or within text, the client sees the fake it sent, so an API that echoes its filters gives nothing away.
The store
One SQLite file on the machine Sosie runs on: ~/.sosie/store/store.db, or under $SOSIE_HOME.
- Keyed hashes, not values. Each real value is stored as an HMAC-SHA256 of it, beside its fake.
- Sealed for lookups. To turn a fake back, the real value is also kept encrypted with AES-256-GCM. It is decrypted only inside Sosie’s process.
- One key.
~/.sosie/store/keyholds 32 random bytes, made on first start, readable only by your user. The hashing, sealing and seeding keys are derived from it. - One store for every upstream. Every source of a Sosie, and every Sosie on the machine, shares it, so a person is the same fake from the database and from the API.
Booleans, single bits, enums and bytes are not stored: their fakes come from the key, and a boolean or a bit a client sends back is turned back by the same flip.
Sources
| Source | Clients speak | Reads only | Names each value from |
|---|---|---|---|
| Postgres | The Postgres wire protocol | Statements are parsed; only reads run, on a read-only session | The table and column Postgres reports for each result column |
| MySQL | MySQL’s protocol: statements as text, and prepared | Statements are parsed as MySQL reads them; only reads run, on a read-only session | The database, table and column MySQL reports for each result column |
| MongoDB | MongoDB’s wire protocol | Commands are allowed by name: reads only, no stage that writes, no JavaScript on the server | The collection and path each value comes from, read from the find or the pipeline |
| SQL Server | TDS, over TLS: batches, parameters and prepared statements | Batches are parsed as SQL Server reads them; only reads and the catalog’s procedures run | The table and column SQL Server names for each value, in browse mode |
| REST | HTTP, at the API’s own paths | GET and HEAD only |
The OpenAPI document, or where each value sits |
| GraphQL | GraphQL over HTTP | Queries only, checked against the schema | The schema, by introspection or a file |
A new database or API is a new source package behind the same interface. Rules, domains, fakes, the store, scan and
describe stay the same for every source.
What it costs
Sosie keeps no copy of your data: it reads the live source on every query, so nothing goes stale, and the store holds only what the fakes need. A large API answer is faked a slice at a time, so other clients are answered meanwhile.