- class dipdup.config.AdvancedConfig(reindex: typing.Dict[dipdup.enums.ReindexingReason, dipdup.enums.ReindexingAction] = <factory>, scheduler: typing.Optional[typing.Dict[str, typing.Any]] = None, postpone_jobs: bool = False, early_realtime: bool = False, merge_subscriptions: bool = False, metadata_interface: bool = False, skip_version_check: bool = False)¶
Feature flags and other advanced config.
- Parameters
reindex – Mapping of reindexing reasons and actions DipDup performs
scheduler – apscheduler scheduler config
postpone_jobs – Do not start job scheduler until all indexes are in realtime state
early_realtime – Establish realtime connection immediately after startup
merge_subscriptions – Subscribe to all operations instead of exact channels
metadata_interface – Expose metadata interface for TzKT
skip_version_check – Do not check for new DipDup versions on startup
- class dipdup.config.BigMapHandlerConfig(callback: str, contract: Union[str, dipdup.config.ContractConfig], path: str)¶
Big map handler config
- Parameters
contract – Contract to fetch big map from
path – Path to big map (alphanumeric string with dots)
- initialize_big_map_type(package: str) None ¶
Resolve imports and initialize key and value type classes
- class dipdup.config.BigMapIndexConfig(kind: Literal['big_map'], datasource: Union[str, dipdup.config.TzktDatasourceConfig], handlers: Tuple[dipdup.config.BigMapHandlerConfig, ...], skip_history: dipdup.enums.SkipHistory = SkipHistory.never, first_level: int = 0, last_level: int = 0)¶
Big map index config
- Parameters
kind – always big_map
datasource – Index datasource to fetch big maps with
handlers – Description of big map diff handlers
skip_history – Fetch only current big map keys ignoring historical changes
first_level – Level to start indexing from
last_level – Level to stop indexing at (Dipdup will terminate at this level)
- class dipdup.config.CallbackMixin(callback: str)¶
Mixin for callback configs
- Parameters
callback – Callback name
- class dipdup.config.CodegenMixin¶
Base for pattern config classes containing methods required for codegen
- locate_arguments() Dict[str, Optional[Type]] ¶
Try to resolve scope annotations for arguments
- class dipdup.config.CoinbaseDatasourceConfig(kind: Literal['coinbase'], api_key: Optional[str] = None, secret_key: Optional[str] = None, passphrase: Optional[str] = None, http: Optional[dipdup.config.HTTPConfig] = None)¶
Coinbase datasource config
- Parameters
kind – always ‘coinbase’
api_key – API key
secret_key – API secret key
passphrase – API passphrase
http – HTTP client configuration
- class dipdup.config.ContractConfig(address: str, typename: Optional[str] = None)¶
Contract config
- Parameters
address – Contract address
typename – User-defined alias for the contract script
- class dipdup.config.DipDupConfig(spec_version: str, package: str, datasources: typing.Dict[str, typing.Union[dipdup.config.TzktDatasourceConfig, dipdup.config.CoinbaseDatasourceConfig, dipdup.config.MetadataDatasourceConfig, dipdup.config.IpfsDatasourceConfig, dipdup.config.HttpDatasourceConfig]], database: typing.Union[dipdup.config.SqliteDatabaseConfig, dipdup.config.PostgresDatabaseConfig] = SqliteDatabaseConfig(kind='sqlite', path=':memory:'), contracts: typing.Dict[str, dipdup.config.ContractConfig] = <factory>, indexes: typing.Dict[str, typing.Union[dipdup.config.OperationIndexConfig, dipdup.config.BigMapIndexConfig, dipdup.config.HeadIndexConfig, dipdup.config.TokenTransferIndexConfig, dipdup.config.IndexTemplateConfig]] = <factory>, templates: typing.Dict[str, typing.Union[dipdup.config.OperationIndexConfig, dipdup.config.BigMapIndexConfig, dipdup.config.HeadIndexConfig, dipdup.config.TokenTransferIndexConfig]] = <factory>, jobs: typing.Dict[str, dipdup.config.JobConfig] = <factory>, hooks: typing.Dict[str, dipdup.config.HookConfig] = <factory>, hasura: typing.Optional[dipdup.config.HasuraConfig] = None, sentry: typing.Optional[dipdup.config.SentryConfig] = None, prometheus: typing.Optional[dipdup.config.PrometheusConfig] = None, advanced: dipdup.config.AdvancedConfig = AdvancedConfig(reindex={}, scheduler=None, postpone_jobs=False, early_realtime=False, merge_subscriptions=False, metadata_interface=False, skip_version_check=False), custom: typing.Dict[str, typing.Any] = <factory>)¶
Main indexer config
- Parameters
spec_version – Version of specification
package – Name of indexer’s Python package, existing or not
datasources – Mapping of datasource aliases and datasource configs
database – Database config
contracts – Mapping of contract aliases and contract configs
indexes – Mapping of index aliases and index configs
templates – Mapping of template aliases and index templates
jobs – Mapping of job aliases and job configs
hooks – Mapping of hook aliases and hook configs
hasura – Hasura integration config
sentry – Sentry integration config
prometheus – Prometheus integration config
advanced – Advanced config
custom – User-defined Custom config
- property oneshot: bool¶
Whether all indexes have last_level field set
- property package_path: str¶
Absolute path to the indexer package, existing or default
- property per_index_rollback: bool¶
Check if package has on_index_rollback hook
- class dipdup.config.HTTPConfig(cache: Optional[bool] = None, retry_count: Optional[int] = None, retry_sleep: Optional[float] = None, retry_multiplier: Optional[float] = None, ratelimit_rate: Optional[int] = None, ratelimit_period: Optional[int] = None, connection_limit: Optional[int] = None, connection_timeout: Optional[int] = None, batch_size: Optional[int] = None)¶
Advanced configuration of HTTP client
- Parameters
cache – Whether to cache responses
retry_count – Number of retries after request failed before giving up
retry_sleep – Sleep time between retries
retry_multiplier – Multiplier for sleep time between retries
ratelimit_rate – Number of requests per period (“drops” in leaky bucket)
ratelimit_period – Time period for rate limiting in seconds
connection_limit – Number of simultaneous connections
connection_timeout – Connection timeout in seconds
batch_size – Number of items fetched in a single paginated request (for some APIs)
- merge(other: Optional[dipdup.config.HTTPConfig]) dipdup.config.HTTPConfig ¶
Set missing values from other config
- class dipdup.config.HandlerConfig(callback: str)¶
- class dipdup.config.HasuraConfig(url: str, admin_secret: Optional[str] = None, source: str = 'default', select_limit: int = 100, allow_aggregations: bool = True, camel_case: bool = False, rest: bool = True, http: Optional[dipdup.config.HTTPConfig] = None)¶
Config for the Hasura integration.
- Parameters
url – URL of the Hasura instance.
admin_secret – Admin secret of the Hasura instance.
source – Hasura source for DipDup to configure, others will be left untouched.
select_limit – Row limit for unauthenticated queries.
allow_aggregations – Whether to allow aggregations in unauthenticated queries.
camel_case – Whether to use camelCase instead of default pascal_case for the field names (incompatible with metadata_interface flag)
rest – Enable REST API both for autogenerated and custom queries.
http – HTTP connection tunables
- property headers: Dict[str, str]¶
Headers to include with every request
- class dipdup.config.HeadHandlerConfig(callback: str)¶
Head block handler config
- class dipdup.config.HeadIndexConfig(kind: Literal['head'], datasource: Union[str, dipdup.config.TzktDatasourceConfig], handlers: Tuple[dipdup.config.HeadHandlerConfig, ...])¶
Head block index config
- class dipdup.config.HookConfig(callback: str, args: typing.Dict[str, str] = <factory>, atomic: bool = False)¶
Hook config
- Parameters
args – Mapping of argument names and annotations (checked lazily when possible)
atomic – Wrap hook in a single database transaction
- class dipdup.config.HttpDatasourceConfig(kind: Literal['http'], url: str, http: Optional[dipdup.config.HTTPConfig] = None)¶
Generic HTTP datasource config
kind: always ‘http’ url: URL to fetch data from http: HTTP client configuration
- class dipdup.config.IndexConfig(kind: str, datasource: Union[str, dipdup.config.TzktDatasourceConfig])¶
Index config
- Parameters
datasource – Alias of index datasource in datasources section
- hash() str ¶
Calculate hash to ensure config has not changed since last run.
- class dipdup.config.IndexTemplateConfig(template: str, values: Dict[str, str], first_level: int = 0, last_level: int = 0)¶
Index template config
- Parameters
kind – always template
name – Name of index template
template_values – Values to be substituted in template (<key> -> value)
first_level – Level to start indexing from
last_level – Level to stop indexing at (DipDup will terminate at this level)
- class dipdup.config.IpfsDatasourceConfig(kind: Literal['ipfs'], url: str = 'https://ipfs.io/ipfs', http: Optional[dipdup.config.HTTPConfig] = None)¶
IPFS datasource config
- Parameters
kind – always ‘ipfs’
url – IPFS node URL, e.g. https://ipfs.io/ipfs/
http – HTTP client configuration
- class dipdup.config.JobConfig(hook: typing.Union[str, dipdup.config.HookConfig], crontab: typing.Optional[str] = None, interval: typing.Optional[int] = None, daemon: bool = False, args: typing.Dict[str, typing.Any] = <factory>)¶
Job schedule config
- Parameters
hook – Name of hook to run
crontab – Schedule with crontab syntax (* * * * *)
interval – Schedule with interval in seconds
daemon – Run hook as a daemon (never stops)
args – Arguments to pass to the hook
- class dipdup.config.LoggingConfig(config: Dict[str, Any])¶
- class dipdup.config.MetadataDatasourceConfig(kind: Literal['metadata'], network: dipdup.datasources.metadata.enums.MetadataNetwork, url: str = 'https://metadata.dipdup.net', http: Optional[dipdup.config.HTTPConfig] = None)¶
DipDup Metadata datasource config
- Parameters
kind – always ‘metadata’
network – Network name, e.g. mainnet, hangzhounet, etc.
url – GraphQL API URL, e.g. https://metadata.dipdup.net
http – HTTP client configuration
- class dipdup.config.NameMixin¶
- class dipdup.config.OperationHandlerConfig(callback: str, pattern: Tuple[Union[dipdup.config.OperationHandlerOriginationPatternConfig, dipdup.config.OperationHandlerTransactionPatternConfig], ...])¶
Operation handler config
- Parameters
callback – Name of method in handlers package
pattern – Filters to match operation groups
- class dipdup.config.OperationHandlerOriginationPatternConfig(type: Literal['origination'] = 'origination', source: Optional[Union[str, dipdup.config.ContractConfig]] = None, similar_to: Optional[Union[str, dipdup.config.ContractConfig]] = None, originated_contract: Optional[Union[str, dipdup.config.ContractConfig]] = None, optional: bool = False, strict: bool = False)¶
Origination handler pattern config
- Parameters
type – always ‘origination’
source – Match operations by source contract alias
similar_to – Match operations which have the same code/signature (depending on strict field)
originated_contract – Match origination of exact contract
optional – Whether can operation be missing in operation group
strict – Match operations by storage only or by the whole code
- class dipdup.config.OperationHandlerTransactionPatternConfig(type: Literal['transaction'] = 'transaction', source: Optional[Union[str, dipdup.config.ContractConfig]] = None, destination: Optional[Union[str, dipdup.config.ContractConfig]] = None, entrypoint: Optional[str] = None, optional: bool = False)¶
Operation handler pattern config
- Parameters
type – always ‘transaction’
source – Match operations by source contract alias
destination – Match operations by destination contract alias
entrypoint – Match operations by contract entrypoint
optional – Whether can operation be missing in operation group
- class dipdup.config.OperationIndexConfig(kind: typing.Literal['operation'], datasource: typing.Union[str, dipdup.config.TzktDatasourceConfig], handlers: typing.Tuple[dipdup.config.OperationHandlerConfig, ...], types: typing.Tuple[dipdup.enums.OperationType, ...] = (<OperationType.transaction: 'transaction'>,), contracts: typing.List[typing.Union[str, dipdup.config.ContractConfig]] = <factory>, first_level: int = 0, last_level: int = 0)¶
Operation index config
- Parameters
kind – always operation
handlers – List of indexer handlers
types – Types of transaction to fetch
contracts – Aliases of contracts being indexed in contracts section
first_level – Level to start indexing from
last_level – Level to stop indexing at (DipDup will terminate at this level)
- property address_filter: Set[str]¶
Set of addresses (any field) to filter operations with before an actual matching
- property entrypoint_filter: Set[Optional[str]]¶
Set of entrypoints to filter operations with before an actual matching
- class dipdup.config.ParameterTypeMixin¶
parameter_type_cls field
- class dipdup.config.ParentMixin¶
parent field for index and template configs
- class dipdup.config.PatternConfig¶
- class dipdup.config.PostgresDatabaseConfig(kind: typing.Literal['postgres'], host: str, user: str = 'postgres', database: str = 'postgres', port: int = 5432, schema_name: str = 'public', password: str = '', immune_tables: typing.Tuple[str, ...] = <factory>, connection_timeout: int = 60)¶
Postgres database connection config
- Parameters
kind – always ‘postgres’
host – Host
port – Port
user – User
password – Password
database – Database name
schema_name – Schema name
immune_tables – List of tables to preserve during reindexing
connection_timeout – Connection timeout
- class dipdup.config.PrometheusConfig(host: str, port: int = 8000, update_interval: float = 1.0)¶
Config for Prometheus integration.
- Parameters
host – Host to bind to
port – Port to bind to
update_interval – Interval to update some metrics in seconds
- class dipdup.config.SentryConfig(dsn: str, environment: Optional[str] = None, debug: bool = False)¶
Config for Sentry integration.
- Parameters
dsn – DSN of the Sentry instance
environment – Environment to report to Sentry (informational only)
debug – Catch warning messages and more context
- class dipdup.config.SqliteDatabaseConfig(kind: Literal['sqlite'], path: str = ':memory:')¶
SQLite connection config
- Parameters
kind – always ‘sqlite’
path – Path to .sqlite3 file, leave default for in-memory database (:memory:)
- class dipdup.config.StorageTypeMixin¶
storage_type_cls field
- class dipdup.config.SubscriptionsMixin¶
subscriptions field
- class dipdup.config.TemplateValuesMixin¶
template_values field
- class dipdup.config.TokenTransferHandlerConfig(callback: str)¶
- class dipdup.config.TokenTransferIndexConfig(kind: typing.Literal['token_transfer'], datasource: typing.Union[str, dipdup.config.TzktDatasourceConfig], handlers: typing.Tuple[dipdup.config.TokenTransferHandlerConfig, ...] = <factory>, first_level: int = 0, last_level: int = 0)¶
Token index config
- class dipdup.config.TransactionIdxMixin¶
transaction_idx field to track index of operation in group
- Parameters
transaction_idx –
- class dipdup.config.TzktDatasourceConfig(kind: Literal['tzkt'], url: str, http: Optional[dipdup.config.HTTPConfig] = None, buffer_size: int = 0)¶
TzKT datasource config
- Parameters
kind – always ‘tzkt’
url – Base API URL, e.g. https://api.tzkt.io/
http – HTTP client configuration
buffer_size – Number of levels to keep in FIFO buffer before processing