Table of contents

Introduction

In this step-by-step guide, you'll set up comprehensive cloud web scraping in Apify using mobile proxies. You'll create an actor in Node.js, run a test task to gather product cards, and learn how to manage limits safely to avoid errors and blocks. This guide is designed for beginners but includes sections for advanced users. By the end, you'll have a reusable actor, a working proxy setup, result verification, checklists, common errors explained, and optimization tips. If you need a quick answer to a practical question, feel free to jump to the FAQ section, but for the best results, follow all the steps.

Who this guide is for: anyone who wants to understand how to launch an Apify actor with mobile proxies without spending weeks sifting through documentation. It'll be beneficial for marketers, analysts, researchers, website owners, and beginner developers who need a reliable and repeatable way to collect data from the web.

What you should know in advance: basic JavaScript concepts will be helpful but are not mandatory. We'll detail where to click, what to input, and how to check results. You should be able to log into the web service, copy access tokens, and handle passwords carefully.

Time required: Expect to spend 2-3 hours for a complete walkthrough, including registration, actor setup, proxy integration, initial testing, and result verification. If you already have an Apify account and access to mobile proxies, you can finish in 60-90 minutes.

Preliminary Preparation

Necessary Tools, Programs, and Access

  • An Apify account with access to run actors.
  • Node.js LTS version (18 or later) installed locally if you want to edit code locally. You can use the built-in editor in Apify, but local development is more convenient.
  • Credentials for mobile proxies. As an example, we’ll use the provider mobileproxy.space, where you can get a username, password, and proxy server address. You may use any similar service.
  • A text editor: such as VS Code or any other.
  • Apify CLI (optional) for local development and uploading actors to the cloud.

System Requirements

  • A stable internet connection.
  • Windows, macOS, or Linux. Any modern computer will work for local actor development.
  • 200-500 MB of free disk space for npm dependencies if you're going through local development.

What to Download and Install

  1. Install Node.js from the official site and choose the LTS version. After installation, check with the terminal command: node -v and npm -v. You should see the versions without errors.
  2. Install the Apify CLI (optional) with the command: npm i -g apify-cli. After installation, verify with: apify --version.
  3. Prepare your mobile proxy credentials: host, port, username, and password. If you're using mobileproxy.space, you'll get an address format like host:port and a user:password pair.

⚠️ Attention: Never publish proxy usernames and passwords in public repositories. Use environment variables or platform secrets.

Creating Backups

If you're editing the code locally, maintain a backup of the project (e.g., using git). The Apify platform keeps versions of actors, but it’s better to have a local backup of the code.

Tip: If you're working with Apify for the first time, start directly in the browser using the editor in the platform's interface, and add local development later. This will speed up your start.

Basic Concepts and What Apify Is

Key Terms in Simple Language

  • Apify — A platform for automating web tasks: scraping, crawling, and integrations. It allows you to run code (actors) in the cloud, store results (Datasets), and manage queues of links.
  • Actor — A containerized application (usually in Node.js or Python) that executes your task: opening pages, gathering data, saving results.
  • Task — A saved configuration for running the actor with pre-filled input. Convenient for regular executions without code changes.
  • Dataset — A storage location for scraping results in table format. Can be exported to JSON, CSV, XLSX.
  • Key-Value Store — A storage for arbitrary files and settings (e.g., input parameters, reports).
  • Request Queue — A queue of links for the crawler, systematically storing and processing URLs.
  • ProxyConfiguration — Proxy settings. You can use Apify proxy or external proxies, including mobile ones.
  • Mobile Proxies — Proxies that use mobile network operators. They are often seen by websites as real mobile traffic.

How It Works

You write an actor, pass input parameters to it, and run it in the cloud. The actor receives a list of links, opens them through the selected crawler (e.g., CheerioCrawler for simple HTML pages or PlaywrightCrawler for complex sites), gathers data, and writes it into the Dataset. For network requests, the actor uses proxies according to ProxyConfiguration. When consistent scraping is needed with low false positives in security, mobile proxies are used. They allow load distribution and make you appear as a mobile user to the target.

What to Understand Before Starting

  • Follow the rules of target websites and applicable laws. Use data collection ethically and legally.
  • Even mobile proxies do not provide immunity from restrictions. Request rate, delays, correct HTTP headers, and crawler code quality matter.
  • Limits of the Apify platform and your subscription plan affect parallelism, memory, and execution time. These can be set and monitored.

Tip: If the target provides an official API, start with that. It is more stable and ethical than scraping HTML.

Step 1: Why Use Mobile Proxies in Cloud Scraping

Goal of the Step

Understand when mobile proxies yield the best results and how to adjust settings to minimize blocks and instability when working from the cloud.

Detailed Step-by-Step Guide

  1. Define your data collection goal: product lists, prices, reviews, schedules, news. Write down specific types of pages and their approximate URLs.
  2. Assess the site's complexity: does the page open without JavaScript, how quickly does it load, is there dynamic loading? If the site is simple, CheerioCrawler is enough; if complex, use PlaywrightCrawler.
  3. Decide if a mobile session is needed: if the site is clearly aimed at mobile users, showing different versions for mobile and desktop clients, mobile proxies will help you appear natural.
  4. Choose a mobile proxy provider. For example, you can use mobileproxy.space. Make sure you have a stable host, port, username, and password. Write them down separately.
  5. Plan your request frequency. Start with 1-2 concurrent tabs and 1-3 requests per second. Gradually increase as needed, watching for errors and responses from the site.
  6. Decide if you'll use IP rotation. For mobile proxies, rotation can occur either on command or via a timer from the provider. Check the policy and rotation commands with your provider.

Important Points

Mobile proxies are suitable when you need to reduce the likelihood of false positives or mimic mobile user behavior. Do not use them for activities prohibited by the website or law. Properly configure User-Agent headers and delays.

⚠️ Warning: Do not attempt to bypass technical restrictions of websites. If a page is closed by authorization or terms of use, act according to the resource's rules.

Expected Outcome

You understand why mobile proxies are used, have chosen a provider, and are ready for configuration in the actor. You have proxy credentials and a planned request frequency.

Possible Issues and Solutions

  • Unclear if a mobile version is needed. Solution: open the site with a mobile User-Agent in the developer's browser and compare the markup. If the difference is significant, a mobile session is appropriate.
  • Uncertain about the provider's reliability. Solution: test the connection through curl with your proxy, checking stability over 10-15 minutes.

✅ Verification: You have accurate proxy parameters (host, port, username, password) and have recorded the desired request frequency.

Step 2: Registering in Apify and Preparing Your Workspace

Goal of the Step

Either create or confirm your Apify account, log into the console, install the Apify CLI if needed, and prepare to create an actor.

Detailed Step-by-Step Guide

  1. Register at Apify. Enter your email, create a password, confirm your e-mail. After logging in, you'll access the console with the Actors, Tasks, and Storage sections.
  2. Go to your profile and find your personal API token. Copy it into a safe place; it will be useful for CLI and integrations.
  3. If using CLI: install apify-cli with the command npm i -g apify-cli. Then run apify login and paste the token. After a successful login, you'll see a confirmation in the terminal.
  4. Create a working folder for your project locally if you're going through local development. Execute apify create and select a Node.js template with Crawlee. A project structure with package.json and src/main.js will be created.
  5. If you're working only in the browser: click New in the Actors section and choose the Node.js + Crawlee template. The platform will create an empty actor and open the online editor.

Important Points

Token security is critical. Do not paste the token into your code. Store it in a password manager. In CLI, it is stored locally and won't get into the repository unless you manually add it.

Tip: Name your actor meaningfully, such as mobile-crawler-products. This will make navigation and automation easier.

Expected Outcome

You are logged into the Apify console, optionally configured CLI, created an empty actor, and see the main.js file in the editor (or locally in the src folder).

Possible Issues and Solutions

  • CLI does not recognize the token. Solution: run apify logout and then apify login again. Ensure you are entering the latest token from your profile.
  • Errors in npm dependency installations. Solution: update Node.js to LTS, clear npm cache with the command npm cache clean --force, and repeat the installation.

✅ Verification: You have created an actor with code editing access and a basic project structure in place.

Step 3: Creating the Actor and Uploading the Template

Goal of the Step

Populate the actor with starter code on Crawlee to immediately launch the crawler and ensure basic functionality without proxies.

Detailed Step-by-Step Guide

  1. Open the main.js file. If it doesn't exist, create src/main.js. Ensure package.json contains dependencies for crawlee and apify.
  2. Insert basic crawler code. Example for CheerioCrawler: import { CheerioCrawler, Dataset } from "crawlee"; export const main = async () => { const startUrls = ["https://example.com/"]; const crawler = new CheerioCrawler({ requestHandler: async ({ request, $, log }) => { const title = $("title").text(); await Dataset.pushData({ url: request.loadedUrl, title }); log.info(`Saved: ${request.loadedUrl}`); }, maxConcurrency: 2, requestHandlerTimeoutSecs: 60, }); for (const url of startUrls) await crawler.addRequests([url]); await crawler.run(); };
  3. Save the file. If in browser, click the Save button. If locally, save changes and run npm install to fetch libraries (if not already fetched).
  4. Try running without proxies: launch the actor with the default input. At least one object with the title field should appear in the Dataset.

Important Points

A minimal MVP for the actor is needed to test the pipeline: launching, logging, and saving results. Before adding proxies, ensure the code works on a simple page.

Tip: Start with one or two initial links. This speeds up tests and simplifies troubleshooting.

Expected Outcome

The actor runs successfully and saves results to the Dataset. Logs indicate data was saved, and there are no errors such as DNS or network timeouts.

Possible Issues and Solutions

  • Package import errors. Solution: check versions in package.json. If necessary, run npm i crawlee apify.
  • No data in the Dataset. Solution: check the selector $("title").text() or replace it with a different simple selection, such as $("h1").first().text().

✅ Verification: The Dataset has at least one object with url and title fields. Logs indicate successful completion with no exceptions.

Step 4: Configuring Proxies in the Actor

Goal of the Step

Connect mobile proxies to the Apify actor so that all network traffic from the crawler goes through the specified proxy server and verify the stability of the connection.

Detailed Step-by-Step Guide

  1. Prepare the proxy string. Format for external HTTP proxy: http://USERNAME:PASSWORD@HOST:PORT. Example: http://user123:pass456@proxy.mobileproxy.space:12345. For mobileproxy.space, use the credentials from your personal account.
  2. Add ProxyConfiguration to the code. For CheerioCrawler: import { CheerioCrawler, Dataset, ProxyConfiguration } from "crawlee"; export const main = async () => { const proxy = new ProxyConfiguration({ proxyUrls: ["http://USERNAME:PASSWORD@HOST:PORT"] }); const crawler = new CheerioCrawler({ proxyConfiguration: proxy, requestHandler: async ({ request, $, log }) => { const title = $("title").text(); await Dataset.pushData({ url: request.loadedUrl, title }); log.info(`Saved: ${request.loadedUrl}`); }, maxConcurrency: 2, requestHandlerTimeoutSecs: 60, }); await crawler.addRequests(["https://httpbin.org/ip"]); await crawler.run(); };
  3. Save changes and run the actor. If everything is correct, in the Dataset you should see the IP address that belongs to your mobile proxy (for httpbin.org/ip this will be JSON with origin or proxy IP).
  4. If using PlaywrightCrawler, add the same ProxyConfiguration in the constructor parameters: import { PlaywrightCrawler, Dataset, ProxyConfiguration } from "crawlee"; const proxy = new ProxyConfiguration({ proxyUrls: ["http://USERNAME:PASSWORD@HOST:PORT"] }); const crawler = new PlaywrightCrawler({ proxyConfiguration: proxy, launchContext: { launchOptions: { headless: true } }, requestHandler: async ({ page, request, log }) => { const title = await page.title(); await Dataset.pushData({ url: request.loadedUrl, title }); log.info(`Saved: ${request.loadedUrl}`); } });
  5. If necessary, move the proxy string to an environment variable and read it via process.env to avoid storing the password in code. On the Apify platform, use the Secrets and ENV Vars sections in the actor's settings. Example: const proxyUrl = process.env.MOBILE_PROXY_URL;

Important Points

Do not mix Apify Proxy and external mobile proxy configurations in the same run. Use one clear proxy source within a single execution. Setting through proxyUrls completely replaces the use of Apify proxy.

Tip: First, test the proxy on simple pages such as https://httpbin.org/ip or similar IP display services. This will quickly confirm that traffic is going through the desired address.

⚠️ Warning: If the mobile proxy provider supports IP rotation via a special URL or command, use this only within their rules. Do not change IP too frequently without necessity: this may raise suspicion on the target website.

Expected Outcome

The crawler is successfully connected to the mobile proxy. When checking the IP (via a status page), you see the proxy address, logs are stable, and requests do not fail due to timeouts.

Possible Issues and Solutions

  • 401 or 407 in logs. Cause: incorrect username or password. Solution: double-check the credentials from the provider's account.
  • ECONNRESET or ETIMEDOUT. Cause: unstable channel or domain blocking. Solution: reduce parallelism, rerun after a pause, check the proxy status with the provider.

✅ Verification: The Dataset contains the result of a request to the IP display page, showing the mobile proxy address.

Step 5: Example Task: Collecting Data from Product Cards

Goal of the Step

Collect data from real product cards using mobile proxies and stable crawler settings, saving the results in the Dataset.

Detailed Step-by-Step Guide

  1. Identify the target list of URLs for cards or categories where you can safely and legally collect open data. Write down 3-5 links for a test.
  2. Select a crawler. If the page is static, use CheerioCrawler. If data loads dynamically, choose PlaywrightCrawler.
  3. Add key selectors to extract data. For example: product name, price, currency, rating, availability. In Cheerio, these will be jQuery-like selectors; in Playwright — page.locator.
  4. Example for CheerioCrawler: import { CheerioCrawler, Dataset, ProxyConfiguration, log } from "crawlee"; export const main = async () => { log.setLevel(log.LEVELS.INFO); const proxy = new ProxyConfiguration({ proxyUrls: [process.env.MOBILE_PROXY_URL] }); const startUrls = ["https://example.com/product/1", "https://example.com/product/2"]; const crawler = new CheerioCrawler({ proxyConfiguration: proxy, maxConcurrency: 2, requestHandlerTimeoutSecs: 90, requestHandler: async ({ request, $, log }) => { const title = $("h1.product-title").text().trim(); const priceText = $("span.price").text().trim(); const availability = $("div.stock").text().trim(); const currency = priceText.replace(/[0-9.,\s]/g, ""); const price = parseFloat(priceText.replace(/[^0-9.,]/g, "").replace(",", ".")) || null; await Dataset.pushData({ url: request.loadedUrl, title, price, currency, availability }); log.info(`Saved: ${title || "no title"}`); }, failedRequestHandler: async ({ request, log }) => { log.warning(`Failed ${request.url}`); } }); await crawler.addRequests(startUrls); await crawler.run(); };
  5. Example for PlaywrightCrawler: import { PlaywrightCrawler, Dataset, ProxyConfiguration } from "crawlee"; export const main = async () => { const proxy = new ProxyConfiguration({ proxyUrls: [process.env.MOBILE_PROXY_URL] }); const startUrls = ["https://example.com/product/1"]; const crawler = new PlaywrightCrawler({ proxyConfiguration: proxy, requestHandlerTimeoutSecs: 120, launchContext: { launchOptions: { headless: true } }, requestHandler: async ({ page, request, log }) => { await page.waitForLoadState("domcontentloaded"); const title = await page.title(); const price = await page.locator("span.price").first().textContent().catch(() => null); await Dataset.pushData({ url: request.loadedUrl, title, price }); log.info(`Saved: ${request.loadedUrl}`); } }); await crawler.addRequests(startUrls); await crawler.run(); };
  6. Save MOBILE_PROXY_URL in the actor's environment variables on the Apify platform (Settings → Environment variables). Value: your proxy string like http://user:pass@host:port.
  7. Run the actor. In the logs, monitor the request status, response time, and number of successfully saved records.

Important Points

The data structure in the Dataset should be consistent: set the same fields for all product cards, otherwise, exporting to tables will be inconvenient. Control timeouts: for dynamic pages, increase requestHandlerTimeoutSecs and add waits for loading key selectors.

Tip: For smooth loads, set min/max delays between requests with autoscaled pool settings or manually add pauses in the handler.

Expected Outcome

The Dataset contains one record per product card with key fields. Logs are stable, with no proxy authorization errors, and the average response time is acceptable for your use case.

Possible Issues and Solutions

  • Incorrect selectors. Cause: responsive layout or different page structure. Solution: check both mobile and desktop versions, use more stable selectors (data attributes, unique IDs).
  • Empty data in certain fields. Cause: values load dynamically. Solution: add explicit waits for needed elements or use Playwright instead of Cheerio.

✅ Verification: The Dataset contains records with url, title, price, or equivalent fields. The average error rate is low and below 5-10% in the test sample.

Step 6: Limits and Optimization

Goal of the Step

Adjust parallelism, timeouts, retries, rotation, and storage to use Apify limits efficiently and enhance collection robustness.

Detailed Step-by-Step Guide

  1. Limit parallelism. Set maxConcurrency in the crawler parameters to between 1 and 3 to start. Gradually increase. The higher the parallelism, the greater the load on proxies and the site.
  2. Configure retries. Crawlee has retryCount and retryTimeoutMillis. Set retryCount = 1-2 to avoid endlessly hitting problematic pages.
  3. Manage execution time. Increase requestHandlerTimeoutSecs to 90-120 for heavy pages. This will reduce false timeouts with slow responses over mobile networks.
  4. Add random delays. Insert small pauses of 300-1500 ms between requests. This appears more natural and reduces the risk of restrictions.
  5. Plan proxy rotation with the provider within reasonable limits. If mobileproxy.space allows requesting a new IP by timer, choose an interval that does not disrupt session stability.
  6. Monitor Apify limits: memory, CPU, time. In the launch settings, specify Memory (e.g., 1024-2048 MB for Playwright) and Max run time (e.g., 30-60 minutes for batches).
  7. Store only necessary fields. The less redundant data in the Dataset, the lower the storage load and faster the export. Remove HTML fragments when unnecessary.
  8. Enable logging at the INFO level and selectively at DEBUG during troubleshooting. Excessive log volume can hinder readability and isn't needed in stable mode.

Important Points

Limit efficiency can be achieved with a few simple rules: low starting parallelism, short retry attempts, precise selectors, and minimizing unnecessary page requests. Monitoring through logs and metrics helps adjust settings.

Tip: Log successful URLs in a Key-Value Store or external storage. This will simplify re-running from a failure point and prevent redundant processing of already collected pages.

Expected Outcome

The actor operates smoothly, does not consume unnecessary resources, and errors occur rarely and predictably. Timeout and parallelism parameters are tailored to the speed of your mobile proxy and website complexity.

Possible Issues and Solutions

  • Increasing timeouts doesn’t help. Cause: page overload or provider issues. Solution: temporarily reduce parallelism to 1 and check connection stability.
  • Too slow speed. Cause: a bottleneck in the proxy network or a heavy site. Solution: increase delays, but also consider splitting tasks into smaller batches.

✅ Verification: The average time per page is stable, the error rate does not increase with higher volume, and memory and time limits are not exceeded.

Result Verification

Checklist: What Should Work

  • The actor runs without errors and completes correctly.
  • Mobile proxies are connected, and IPs in test requests correspond to the proxy.
  • The Dataset contains expected fields and values.
  • Logs are informative but not overloaded.
  • When re-running, there are no unnecessary duplicates (or they are controlled).

How to Test

  1. Run the actor with 2-3 test URLs with the proxy enabled and check the IP via a status page.
  2. Compare numbers: how many requests were added and how many results you received. They should match or differ within a reasonable error margin.
  3. Export the Dataset to CSV and ensure the data is clean: no nulls where you expect values.

Success Metrics

  • The failure percentage is below 5-10% during testing.
  • The average page processing time is stable and predictable.
  • There are no abnormal spikes in timeouts and proxy authorization errors.

Tip: Keep a control set of URLs and repeat tests before each major code change. This way, you'll quickly catch regressions.

Common Errors and Solutions

  • Issue: 407 Proxy Authentication Required. Cause: incorrect proxy credentials. Solution: double-check your username and password, update environment variables, and restart the actor.
  • Issue: ECONNRESET and ETIMEDOUT in logs. Cause: network instability or overload. Solution: lower maxConcurrency, increase timeouts, and add pauses between requests.
  • Issue: empty fields in the Dataset. Cause: incorrect selectors or dynamic loading. Solution: use PlaywrightCrawler, add waits, and reevaluate selectors.
  • Issue: memory limit reached. Cause: too many parallel tabs or storing excessive data. Solution: reduce parallelism, cut down on data volume, and increase Memory in launch settings.
  • Issue: too slow collection. Cause: heavy pages and mobile networks. Solution: prioritize data queue importance, split tasks into batches, optimize selectors, and disable unnecessary navigation.
  • Issue: duplicates in results. Cause: re-running with the same URLs without filtering. Solution: maintain a processed links list in the Request Queue with uniqueness, or check for duplicates before saving.
  • Issue: sensitive site reacts to frequent requests. Cause: overly aggressive pacing. Solution: reduce speed, add jitter to delays, use correct headers, and an up-to-date User-Agent.

Tip: During debugging, temporarily enable detailed logs for one or two URLs and analyze each step. This is quicker than dealing with lengthy batches.

Additional Opportunities

Advanced Settings

  • Secrets and configurations. Store MOBILE_PROXY_URL and other keys in the Secrets section. Read them in the code via process.env.
  • Changing User-Agent. To simulate a mobile client, set a mobile User-Agent and corresponding viewport width in Playwright. Do this moderately and only when needed for correct page rendering.
  • Task scheduler. Create a Task and set up a schedule for runs (daily, hourly). Monitor limits and result volumes.

Optimization

  • Caching. If pages change rarely, add request and revisit caching to not waste proxies and limits unnecessarily.
  • Queues and priorities. Work through Request Queue, assigning higher priority to important links while skipping secondary ones.
  • Splitting into microservices. Divide a complex task into several actors: collecting links, processing cards, validation, and exportation.

What Else Can Be Done

  • API integrations. Connect to send results to your CRM or analytics system after each run via Webhook.
  • Data validation. Before export, check schemas to ensure all values match expected types and ranges.
  • Internal document links. Refer back to the Limits and Optimization section when adjusting performance.

Tip: Use preview mode and small batches for initial runs in the schedule, then scale up gradually.

FAQ

  • How can I tell if a proxy is truly mobile? Check ASN and network type by IP through third-party databases and compare it with mobile operators. Mobile proxies often have a characteristic address pool with dynamic changes.
  • Can I use multiple mobile proxies at once? Yes, specify multiple proxyUrls. Crawlee will automatically select one from the list. Monitor the limits of each proxy.
  • What to do if the website shows a captcha? Reduce frequency, add delays, check headers, and consider using the site's official API. Avoid actions that violate site rules.
  • How to store proxy passwords safely? Use environment variables and Secrets on the Apify platform. Do not commit passwords to git.
  • Do I need to change the User-Agent to mobile? Only if the website serves different versions of the page. In other cases, stable behavior and correct delays are sufficient.
  • Why is CheerioCrawler faster? It parses HTML rather than rendering the page. For dynamic pages, use PlaywrightCrawler, though it's slower.
  • How to export results? In the Dataset interface, select export to CSV, JSON, XLSX. Or use Datasets API if you want to automate exports.
  • Can I combine Apify Proxy and mobile proxies? It's best to use one or the other in a single run. If you need different sources, separate tasks by actor or launch configurations.
  • How many requests per second is safe? Start with 1-3 per second and monitor metrics. For sensitive sites, reduce to 0.2-0.5 with pauses.
  • Do I need to enable headful in Playwright? Only for debugging. In production, use headless to save resources.

Conclusion

You have set up a working Apify actor with mobile proxies, understood key concepts and principles, collected test data from product cards, and optimized limits. You now confidently manage parallelism, wait time, and result storage, as well as how to secure secrets and passwords. Moving forward, you can expand the project: add new types of pages, build a pipeline of multiple actors, connect a scheduler, and automate exports. If specific questions arise, refer back to the FAQ section or the Limits and Optimization section. Remember, mobile proxies are a tool to enhance traffic stability and naturalness, not a way to circumvent restrictions. Work ethically, follow website rules, and always start small, checking each change.

About the Author

Roman Melnikov

Roman Melnikov

Technical Writer and System Administrator

Work Experience: Technical writer and DevOps engineer with 9 years of experience. Created over 50 detailed guides on system configuration and administration. His instructions helped thousands of professionals successfully solve technical tasks. Popular author on Habr and YouTube.
Education: Bauman Moscow State Technical University. Information Systems and Technologies
Expertise:
Technical Documentation DevOps System Administration Linux Docker and Kubernetes CI/CD Infrastructure Automation Cloud Technologies System Monitoring Bash and Python Scripting

Share this article: