Console/Ingest & scale
operational
Ingest & scale
Pour sites into the frontier, then run the bounded queue worker. Every worker bound is yours to set — batch size, crawl depth, page budget, and corpus ceiling.
Pour sites into the frontier, then run the bounded queue worker. Every worker bound is yours to set — batch size, crawl depth, page budget, and corpus ceiling.