class dipdup.config.AdvancedConfig(reindex: typing.Dict[dipdup.enums.ReindexingReason, dipdup.enums.ReindexingAction] = <factory>, scheduler: typing.Optional[typing.Dict[str, typing.Any]] = None, postpone_jobs: bool = False, early_realtime: bool = False, merge_subscriptions: bool = False, metadata_interface: bool = False, skip_version_check: bool = False)

Feature flags and other advanced config.

Parameters

  • reindex – Mapping of reindexing reasons and actions DipDup performs

  • scheduler – apscheduler scheduler config

  • postpone_jobs – Do not start job scheduler until all indexes are in realtime state

  • early_realtime – Establish realtime connection immediately after startup

  • merge_subscriptions – Subscribe to all operations instead of exact channels

  • metadata_interface – Expose metadata interface for TzKT

  • skip_version_check – Do not check for new DipDup versions on startup

class dipdup.config.BigMapHandlerConfig(callback: str, contract: Union[str, dipdup.config.ContractConfig], path: str)

Big map handler config

Parameters

  • contract – Contract to fetch big map from

  • path – Path to big map (alphanumeric string with dots)

initialize_big_map_type(package: str) → None

Resolve imports and initialize key and value type classes

class dipdup.config.BigMapIndexConfig(kind: Literal['big_map'], datasource: Union[str, dipdup.config.TzktDatasourceConfig], handlers: Tuple[dipdup.config.BigMapHandlerConfig, ...], skip_history: dipdup.enums.SkipHistory = SkipHistory.never, first_level: int = 0, last_level: int = 0)

Big map index config

Parameters

  • kind – always big_map

  • datasource – Index datasource to fetch big maps with

  • handlers – Description of big map diff handlers

  • skip_history – Fetch only current big map keys ignoring historical changes

  • first_level – Level to start indexing from

  • last_level – Level to stop indexing at (Dipdup will terminate at this level)

class dipdup.config.CallbackMixin(callback: str)

Mixin for callback configs

Parameters

callback – Callback name

class dipdup.config.CodegenMixin

Base for pattern config classes containing methods required for codegen

locate_arguments() → Dict[str, Optional[Type]]

Try to resolve scope annotations for arguments

class dipdup.config.CoinbaseDatasourceConfig(kind: Literal['coinbase'], api_key: Optional[str] = None, secret_key: Optional[str] = None, passphrase: Optional[str] = None, http: Optional[dipdup.config.HTTPConfig] = None)

Coinbase datasource config

Parameters

  • kind – always ‘coinbase’

  • api_key – API key

  • secret_key – API secret key

  • passphrase – API passphrase

  • http – HTTP client configuration

class dipdup.config.ContractConfig(address: str, typename: Optional[str] = None)

Contract config

Parameters

  • address – Contract address

  • typename – User-defined alias for the contract script

class dipdup.config.DipDupConfig(spec_version: str, package: str, datasources: typing.Dict[str, typing.Union[dipdup.config.TzktDatasourceConfig, dipdup.config.CoinbaseDatasourceConfig, dipdup.config.MetadataDatasourceConfig, dipdup.config.IpfsDatasourceConfig, dipdup.config.HttpDatasourceConfig]], database: typing.Union[dipdup.config.SqliteDatabaseConfig, dipdup.config.PostgresDatabaseConfig] = SqliteDatabaseConfig(kind='sqlite', path=':memory:'), contracts: typing.Dict[str, dipdup.config.ContractConfig] = <factory>, indexes: typing.Dict[str, typing.Union[dipdup.config.OperationIndexConfig, dipdup.config.BigMapIndexConfig, dipdup.config.HeadIndexConfig, dipdup.config.TokenTransferIndexConfig, dipdup.config.IndexTemplateConfig]] = <factory>, templates: typing.Dict[str, typing.Union[dipdup.config.OperationIndexConfig, dipdup.config.BigMapIndexConfig, dipdup.config.HeadIndexConfig, dipdup.config.TokenTransferIndexConfig]] = <factory>, jobs: typing.Dict[str, dipdup.config.JobConfig] = <factory>, hooks: typing.Dict[str, dipdup.config.HookConfig] = <factory>, hasura: typing.Optional[dipdup.config.HasuraConfig] = None, sentry: typing.Optional[dipdup.config.SentryConfig] = None, prometheus: typing.Optional[dipdup.config.PrometheusConfig] = None, advanced: dipdup.config.AdvancedConfig = AdvancedConfig(reindex={}, scheduler=None, postpone_jobs=False, early_realtime=False, merge_subscriptions=False, metadata_interface=False, skip_version_check=False), custom: typing.Dict[str, typing.Any] = <factory>)

Main indexer config

Parameters

  • spec_version – Version of specification

  • package – Name of indexer’s Python package, existing or not

  • datasources – Mapping of datasource aliases and datasource configs

  • database – Database config

  • contracts – Mapping of contract aliases and contract configs

  • indexes – Mapping of index aliases and index configs

  • templates – Mapping of template aliases and index templates

  • jobs – Mapping of job aliases and job configs

  • hooks – Mapping of hook aliases and hook configs

  • hasura – Hasura integration config

  • sentry – Sentry integration config

  • prometheus – Prometheus integration config

  • advanced – Advanced config

  • custom – User-defined Custom config

property oneshot: bool

Whether all indexes have last_level field set

property package_path: str

Absolute path to the indexer package, existing or default

property per_index_rollback: bool

Check if package has on_index_rollback hook

class dipdup.config.HTTPConfig(cache: Optional[bool] = None, retry_count: Optional[int] = None, retry_sleep: Optional[float] = None, retry_multiplier: Optional[float] = None, ratelimit_rate: Optional[int] = None, ratelimit_period: Optional[int] = None, connection_limit: Optional[int] = None, connection_timeout: Optional[int] = None, batch_size: Optional[int] = None)

Advanced configuration of HTTP client

Parameters

  • cache – Whether to cache responses

  • retry_count – Number of retries after request failed before giving up

  • retry_sleep – Sleep time between retries

  • retry_multiplier – Multiplier for sleep time between retries

  • ratelimit_rate – Number of requests per period (“drops” in leaky bucket)

  • ratelimit_period – Time period for rate limiting in seconds

  • connection_limit – Number of simultaneous connections

  • connection_timeout – Connection timeout in seconds

  • batch_size – Number of items fetched in a single paginated request (for some APIs)

merge(other: Optional[dipdup.config.HTTPConfig]) → dipdup.config.HTTPConfig

Set missing values from other config

class dipdup.config.HandlerConfig(callback: str)
class dipdup.config.HasuraConfig(url: str, admin_secret: Optional[str] = None, source: str = 'default', select_limit: int = 100, allow_aggregations: bool = True, camel_case: bool = False, rest: bool = True, http: Optional[dipdup.config.HTTPConfig] = None)

Config for the Hasura integration.

Parameters

  • url – URL of the Hasura instance.

  • admin_secret – Admin secret of the Hasura instance.

  • source – Hasura source for DipDup to configure, others will be left untouched.

  • select_limit – Row limit for unauthenticated queries.

  • allow_aggregations – Whether to allow aggregations in unauthenticated queries.

  • camel_case – Whether to use camelCase instead of default pascal_case for the field names (incompatible with metadata_interface flag)

  • rest – Enable REST API both for autogenerated and custom queries.

  • http – HTTP connection tunables

property headers: Dict[str, str]

Headers to include with every request

class dipdup.config.HeadHandlerConfig(callback: str)

Head block handler config

class dipdup.config.HeadIndexConfig(kind: Literal['head'], datasource: Union[str, dipdup.config.TzktDatasourceConfig], handlers: Tuple[dipdup.config.HeadHandlerConfig, ...])

Head block index config

class dipdup.config.HookConfig(callback: str, args: typing.Dict[str, str] = <factory>, atomic: bool = False)

Hook config

Parameters

  • args – Mapping of argument names and annotations (checked lazily when possible)

  • atomic – Wrap hook in a single database transaction

class dipdup.config.HttpDatasourceConfig(kind: Literal['http'], url: str, http: Optional[dipdup.config.HTTPConfig] = None)

Generic HTTP datasource config

kind: always ‘http’ url: URL to fetch data from http: HTTP client configuration

class dipdup.config.IndexConfig(kind: str, datasource: Union[str, dipdup.config.TzktDatasourceConfig])

Index config

Parameters

datasource – Alias of index datasource in datasources section

hash() → str

Calculate hash to ensure config has not changed since last run.

class dipdup.config.IndexTemplateConfig(template: str, values: Dict[str, str], first_level: int = 0, last_level: int = 0)

Index template config

Parameters

  • kind – always template

  • name – Name of index template

  • template_values – Values to be substituted in template (<key> -> value)

  • first_level – Level to start indexing from

  • last_level – Level to stop indexing at (DipDup will terminate at this level)

class dipdup.config.IpfsDatasourceConfig(kind: Literal['ipfs'], url: str = 'https://ipfs.io/ipfs', http: Optional[dipdup.config.HTTPConfig] = None)

IPFS datasource config

Parameters

  • kind – always ‘ipfs’

  • url – IPFS node URL, e.g. https://ipfs.io/ipfs/

  • http – HTTP client configuration

class dipdup.config.JobConfig(hook: typing.Union[str, dipdup.config.HookConfig], crontab: typing.Optional[str] = None, interval: typing.Optional[int] = None, daemon: bool = False, args: typing.Dict[str, typing.Any] = <factory>)

Job schedule config

Parameters

  • hook – Name of hook to run

  • crontab – Schedule with crontab syntax (* * * * *)

  • interval – Schedule with interval in seconds

  • daemon – Run hook as a daemon (never stops)

  • args – Arguments to pass to the hook

class dipdup.config.LoggingConfig(config: Dict[str, Any])
class dipdup.config.MetadataDatasourceConfig(kind: Literal['metadata'], network: dipdup.datasources.metadata.enums.MetadataNetwork, url: str = 'https://metadata.dipdup.net', http: Optional[dipdup.config.HTTPConfig] = None)

DipDup Metadata datasource config

Parameters

  • kind – always ‘metadata’

  • network – Network name, e.g. mainnet, hangzhounet, etc.

  • url – GraphQL API URL, e.g. https://metadata.dipdup.net

  • http – HTTP client configuration

class dipdup.config.NameMixin
class dipdup.config.OperationHandlerConfig(callback: str, pattern: Tuple[Union[dipdup.config.OperationHandlerOriginationPatternConfig, dipdup.config.OperationHandlerTransactionPatternConfig], ...])

Operation handler config

Parameters

  • callback – Name of method in handlers package

  • pattern – Filters to match operation groups

class dipdup.config.OperationHandlerOriginationPatternConfig(type: Literal['origination'] = 'origination', source: Optional[Union[str, dipdup.config.ContractConfig]] = None, similar_to: Optional[Union[str, dipdup.config.ContractConfig]] = None, originated_contract: Optional[Union[str, dipdup.config.ContractConfig]] = None, optional: bool = False, strict: bool = False)

Origination handler pattern config

Parameters

  • type – always ‘origination’

  • source – Match operations by source contract alias

  • similar_to – Match operations which have the same code/signature (depending on strict field)

  • originated_contract – Match origination of exact contract

  • optional – Whether can operation be missing in operation group

  • strict – Match operations by storage only or by the whole code

class dipdup.config.OperationHandlerTransactionPatternConfig(type: Literal['transaction'] = 'transaction', source: Optional[Union[str, dipdup.config.ContractConfig]] = None, destination: Optional[Union[str, dipdup.config.ContractConfig]] = None, entrypoint: Optional[str] = None, optional: bool = False)

Operation handler pattern config

Parameters

  • type – always ‘transaction’

  • source – Match operations by source contract alias

  • destination – Match operations by destination contract alias

  • entrypoint – Match operations by contract entrypoint

  • optional – Whether can operation be missing in operation group

class dipdup.config.OperationIndexConfig(kind: typing.Literal['operation'], datasource: typing.Union[str, dipdup.config.TzktDatasourceConfig], handlers: typing.Tuple[dipdup.config.OperationHandlerConfig, ...], types: typing.Tuple[dipdup.enums.OperationType, ...] = (<OperationType.transaction: 'transaction'>,), contracts: typing.List[typing.Union[str, dipdup.config.ContractConfig]] = <factory>, first_level: int = 0, last_level: int = 0)

Operation index config

Parameters

  • kind – always operation

  • handlers – List of indexer handlers

  • types – Types of transaction to fetch

  • contracts – Aliases of contracts being indexed in contracts section

  • first_level – Level to start indexing from

  • last_level – Level to stop indexing at (DipDup will terminate at this level)

property address_filter: Set[str]

Set of addresses (any field) to filter operations with before an actual matching

property entrypoint_filter: Set[Optional[str]]

Set of entrypoints to filter operations with before an actual matching

class dipdup.config.ParameterTypeMixin

parameter_type_cls field

class dipdup.config.ParentMixin

parent field for index and template configs

class dipdup.config.PatternConfig
class dipdup.config.PostgresDatabaseConfig(kind: typing.Literal['postgres'], host: str, user: str = 'postgres', database: str = 'postgres', port: int = 5432, schema_name: str = 'public', password: str = '', immune_tables: typing.Tuple[str, ...] = <factory>, connection_timeout: int = 60)

Postgres database connection config

Parameters

  • kind – always ‘postgres’

  • host – Host

  • port – Port

  • user – User

  • password – Password

  • database – Database name

  • schema_name – Schema name

  • immune_tables – List of tables to preserve during reindexing

  • connection_timeout – Connection timeout

class dipdup.config.PrometheusConfig(host: str, port: int = 8000, update_interval: float = 1.0)

Config for Prometheus integration.

Parameters

  • host – Host to bind to

  • port – Port to bind to

  • update_interval – Interval to update some metrics in seconds

class dipdup.config.SentryConfig(dsn: str, environment: Optional[str] = None, debug: bool = False)

Config for Sentry integration.

Parameters

  • dsn – DSN of the Sentry instance

  • environment – Environment to report to Sentry (informational only)

  • debug – Catch warning messages and more context

class dipdup.config.SqliteDatabaseConfig(kind: Literal['sqlite'], path: str = ':memory:')

SQLite connection config

Parameters

  • kind – always ‘sqlite’

  • path – Path to .sqlite3 file, leave default for in-memory database (:memory:)

class dipdup.config.StorageTypeMixin

storage_type_cls field

class dipdup.config.SubscriptionsMixin

subscriptions field

class dipdup.config.TemplateValuesMixin

template_values field

class dipdup.config.TokenTransferHandlerConfig(callback: str)
class dipdup.config.TokenTransferIndexConfig(kind: typing.Literal['token_transfer'], datasource: typing.Union[str, dipdup.config.TzktDatasourceConfig], handlers: typing.Tuple[dipdup.config.TokenTransferHandlerConfig, ...] = <factory>, first_level: int = 0, last_level: int = 0)

Token index config

class dipdup.config.TransactionIdxMixin

transaction_idx field to track index of operation in group

Parameters

transaction_idx –

class dipdup.config.TzktDatasourceConfig(kind: Literal['tzkt'], url: str, http: Optional[dipdup.config.HTTPConfig] = None, buffer_size: int = 0)

TzKT datasource config

Parameters

  • kind – always ‘tzkt’

  • url – Base API URL, e.g. https://api.tzkt.io/

  • http – HTTP client configuration

  • buffer_size – Number of levels to keep in FIFO buffer before processing