Skip to content

Contact Ingestion Overview

Scope

This page summarizes the contact-ingestion paths in cred-api-commercial, cred-web-commercial, and cred-ios-commercial.

Executive Summary

Contacts enter CRED through two primary paths:

Path Purpose Primary primitives
Bulk contact import Ingest external datasets with source-field mapping, row materialization, and optional reconcile/comparison Import, ImportField, ImportRecord
Bulk create contacts Create contacts asynchronously from client-supplied payloads, with job polling and optional cleanup startBulkCreateContacts, bulkCreateContactsJob, startBulkDeleteCreatedContacts, bulkDeleteCreatedContactsJob

Both paths are production-ready. Choose based on the shape of your source data and the UX you need.

When to Use Which Path

flowchart TD
    Start{"What kind of contact ingestion?"}

    ImportPath[Bulk Import]
    CreatePath[Bulk Create]

    ImportCases["File upload, CRM sync,\nwebhook, Universal API,\nor Polytomic ingestion"]
    CreateCases["iOS device contacts,\nclient-driven batch creation,\nor any app-supplied contact list"]

    ImportDoc[See Bulk Contact Import]
    CreateDoc[See Bulk Create & Cleanup]

    Start -- "External dataset with\nfield mapping or reconcile needs" --> ImportPath
    Start -- "Client-supplied contact objects\nwith async job UX" --> CreatePath
    ImportPath --> ImportCases --> ImportDoc
    CreatePath --> CreateCases --> CreateDoc

See Bulk Contact Import and Bulk Create & Cleanup for full details on each path.

Use bulk import when you need:

  • source-field mapping
  • persisted row materialization
  • reconcile workflows
  • comparison or import provenance
  • CRM/file/webhook/Universal API style ingestion

Use bulk create when you need:

  • client-supplied contact objects
  • async create UX with job polling
  • a fast path for "make these contacts appear in a list"
  • an operator/test cleanup path after creation

iOS Device Contact Flow

The iOS device contact sync uses the bulk-create path:

  1. iOS resolves or creates a "Device Contacts" collection
  2. iOS calls startBulkCreateContacts
  3. iOS polls bulkCreateContactsJob(jobId)
  4. Worker creates contacts and associates them with the collection
  5. iOS shows the populated Device Contacts list
  6. Operator runs startBulkDeleteCreatedContacts(dryRun: true) through cred-platform query --env local
  7. Operator verifies candidateContactIds
  8. Operator runs startBulkDeleteCreatedContacts(dryRun: false)
  9. Worker deletes the tracked contacts
  10. iOS refresh shows the Device Contacts list empty again

Call Flow and Job Types

Create

  • Start mutation: startBulkCreateContacts
  • Status query: bulkCreateContactsJob
  • Worker task: BULK_CREATE_CONTACTS

Delete

  • Start mutation: startBulkDeleteCreatedContacts
  • Status query: bulkDeleteCreatedContactsJob
  • Worker task: BULK_DELETE_CREATED_CONTACTS

Some successful create/delete runs also enqueue follow-up tasks such as SET_PERSON_COMMERCIAL_DATA. Those are downstream side effects, not evidence that the bulk create/delete job itself failed.

Identifiers That Matter

For each test or support run, keep:

  • collectionId
  • bulk create jobId
  • bulk create trackingKey
  • bulk delete dry-run jobId
  • bulk delete real jobId

What each identifier means

Identifier Scope Durability Why it matters
jobId One async create/delete job Cache-backed, 24h TTL Polling and log correlation
trackingKey One bulk-create run Persisted onto contacts Cleanup and recovery
collectionId Target contact collection Persisted in DB UI verification and auth behavior

Durable Contact Marker

Bulk-created contacts are stamped with:

  • Contact.externalSource = trackingKey

Current tracking keys look like:

  • bcc:j:<bulkCreateJobId>
  • optionally bcc:j:<bulkCreateJobId>:s:<clientSource>:r:<clientReferenceId>

That marker is the durable recovery hook for "which contacts came from this run?"

Observability and Recovery

Normal operator flow

Use the GraphQL job queries for normal create/delete polling:

  • bulkCreateContactsJob(jobId)
  • bulkDeleteCreatedContactsJob(jobId)

Recent jobs

Recent create/delete jobs live in Redis cache for 24 hours.

Useful Redis key patterns:

  • bulk-create-contacts:job:<jobId>
  • bulk-create-contacts:idempotency:<userId>:<idempotencyKey>
  • bulk-delete-created-contacts:job:<jobId>

Older create runs

Once cache expires, bulk-create runs are still recoverable from Contact.externalSource like 'bcc:j:%'.

That is the durable query surface for:

  • forgotten jobs
  • orphaned test data
  • support cleanup

Older delete runs

Delete jobs do not currently leave a comparable durable history marker once cache expires. After that, the signal is:

  • logs
  • whether tracked contacts still exist

Expected States

Create and delete jobs both use:

  • QUEUED
  • PROCESSING
  • COMPLETED
  • PARTIAL
  • FAILED

Operationally:

  • COMPLETED means the requested work succeeded
  • PARTIAL means some rows/entities succeeded and some did not
  • FAILED means the job itself failed at a higher level

Expected Errors and Failure Modes

Collection authorization failure

Observed error:

  • FORBIDDEN: Not authorized to update the collection

Meaning:

  • create was pointed at a collection the current caller could not edit
  • the mutation-side auth check is functioning correctly

The validated iOS fix was to resolve/create a device-contacts collection editable by the current user instead of reusing a same-title collection owned by someone else.

Job stuck at QUEUED

Historical local causes included:

  • no real worker process
  • local env accidentally forcing mock-worker behavior

Current expectation is that the top-level local startup wrapper brings up the worker automatically.

Local Tools

Start the stack

cd <your-workspace>
./start-local-federated-development.sh

Tail logs

cd <your-workspace>/api/cred-api-commercial
./scripts/tail-local-bulk-contact-logs.sh \
  --pattern 'bulk-create-contacts|bulk-delete-created-contacts|jobId|trackingKey|bcc:j:'

Run authenticated local GraphQL

cd <your-workspace>/mobile/cred-ios-commercial
./cred-platform query --env local 'query { __typename }'

The current local operator path intentionally uses cred-platform query --env local for cleanup and job inspection.

Current Documentation Map