Senior Web Scraping Architect & Python Data Engineer (Master-Level Only)

Please login or register as jobseeker to apply for this job.

: TYPE OF WORK

Part Time

: SALARY

$7-8 hour

: HOURS PER WEEK

20

: DATE UPDATED

Sep 9, 2026

JOB OVERVIEW

Position: Part-Time (Scaling to Full-Time based on performance)
Starting Rate: $7.00 - $8.00 USD / hour (Rapid increases for true experts)

Please read carefully: This is NOT an entry-level or standard software engineering role.

We are looking for an elite, battle-tested Web Scraping Master. Our internal Software Lead already builds highly effective scrapers, so we do not need a junior developer to write basic HTML parsing scripts. We are bringing on a heavy-hitting specialist to tackle our most complex extraction targets, scale our infrastructure, and smash through advanced anti-bot roadblocks.

If you are a generalist Python developer who occasionally uses Selenium or Scrapy, this role is not for you. We need someone who dreams in network tabs, reverse-engineers undocumented APIs, and bypasses enterprise-grade bot protection for breakfast.

What You Will Do
Architect, deploy, and scale high-volume data extraction pipelines.

Partner directly with our US Lead Programmer to solve our most aggressive scraping roadblocks and optimize existing systems.

Reverse-engineer complex mobile and web APIs, intercepting network traffic to extract data without rendering the DOM whenever possible.

Bypass advanced anti-bot mitigations (Cloudflare Turnstile, DataDome, PerimeterX, Akamai) using modern evasion techniques.

Manage and scale your own infrastructure, utilizing VM clusters, containerization, and advanced proxy rotation (residential, mobile, ISP) to maintain high success rates.

The Minimum Requirements
Mastery of Python Scraping: Deep, production-level experience with advanced frameworks (Scrapy, Playwright, Puppeteer-stealth, undetected-chromedriver, TLS manipulation).

Anti-Bot Evasion: Proven ability to handle browser fingerprinting, canvas fingerprinting, TLS/JA3 spoofing, and behavioral analysis mitigation.

Independent Infrastructure: You must already possess the VMs, hardware, and scalable architecture to pull massive datasets quickly and accurately.

Network Analysis: Expert at digging into Chrome DevTools, reading XHR/Fetch requests, and reconstructing hidden API calls (including handling protobufs or encrypted payloads).

Impeccable English & Technical Communication: You must be able to clearly communicate complex technical architectures and bottleneck solutions to our US-based team.

Who Should NOT Apply
General software engineers looking for side work.

Developers whose scraping experience is limited to standard HTML parsing (BeautifulSoup) on unprotected sites.

Anyone who relies solely on expensive, third-party "scraper APIs" instead of knowing how to build the bypasses themselves.

How to Apply (Mandatory Technical Screen)
To prove you operate at the level we require, you must answer the following 3 scenarios in your cover letter. We will not review applications that do not include these answers. Our Lead Programmer will be rigorously evaluating your methodology.

The TLS Problem: You are targeting a site heavily protected by Cloudflare. Your residential proxies are clean, and your headers are perfect, but your Python requests are instantly flagged. Explain how TLS/JA3 fingerprinting is likely exposing you and exactly what tools/libraries you would use to spoof it.

The API Puzzle: You find that a target site pulls its core data via a GraphQL API. However, the API requests require a dynamic cryptographic token in the header that is generated by a heavily obfuscated JavaScript file on page load. Walk us through your exact process for reverse-engineering this token generation so you can hit the API directly without running a headless browser.

Infrastructure & Scale: Describe the VM and proxy architecture you currently use for tasks requiring 1M+ requests per day. How do you handle queue management, retry logic, and proxy burning?

Please also confirm your available working hours in Philippine Standard Time (PHT).

SKILL REQUIREMENT

Python REST API Web Scraping

SHARE THIS POST

Job Post	Reporter	Rank	Reason	Date

BENCHMARKS

Loading Time: Base Classes	0.0011
Controller Execution Time ( Jobseekers / Job )	0.0168
Total Execution Time	0.0188

GET DATA

No GET data exists

MEMORY USAGE

1,504,168 bytes

POST DATA

No POST data exists

URI STRING

jobseekers/job/Senior-Web-Scraping-Architect-Python-Data-Engineer-Master-Level-Only-1726985

CLASS/METHOD

jobseekers/job

DATABASE: onlinejobs (Jobseekers:$db) QUERIES: 13 (0.0098 seconds) (Hide)

0.0004	`SELECT * FROM exrates WHERE rate_name = 'USD-PHP'`
0.0004	SELECT * FROM `employer_jobs` WHERE `job_id` = 1726985 LIMIT 1
0.0008	SELECT * FROM `employers` WHERE `employer_id` = 884547 LIMIT 1
0.0008	SELECT COUNT(*) AS `numrows` FROM `t_thread` `t` LEFT JOIN `t_thread_misc` `misc` ON `t`.`id` = `misc`.`thread_id` WHERE `t`.`job_id` = 1726985 AND `misc`.`id` IS NULL
0.0005	SELECT e.business_name, e.logo, e.website, e.rebill_date, e.date_added member_date, hits, DATEDIFF('2026-09-27',ej.date_added) duration_days, DATEDIFF('2026-09-27',e.rebill_date) duration_rebill, ej.*, e.deactivate FROM employers e, employer_jobs ej WHERE e.employer_id = ej.employer_id AND ((e.user_level >= '500' AND ej.date_added <= e.rebill_date) OR e.employer_id = '' OR (ej.date_approved <> '2000-01-01' and DATEDIFF('2026-09-27',ej.date_added) <= 14 )) AND e.deactivate != 1 AND ej.deleted = 0 AND job_id = '1726985'
0.0003	SELECT * FROM `employer_jobs_skills` `ejs` LEFT JOIN `skills_categories` `sc` ON `ejs`.`skill_id` = `sc`.`id` WHERE `job_id` = 1726985
0.0006	`UPDATE employer_jobs SET hit_counts = '*Sep-09-2026=283Sep-10-2026=125Sep-11-2026=39Sep-12-2026=19Sep-13-2026=13Sep-14-2026=19Sep-15-2026=26Sep-16-2026=23Sep-17-2026=9Sep-18-2026=15Sep-19-2026=11Sep-20-2026=12Sep-21-2026=12Sep-22-2026=4Sep-23-2026=6Sep-24-2026=7Sep-25-2026=8Sep-26-2026=5**Sep-27-2026=2' WHERE job_id= '1726985'`
0.0005	`UPDATE employer_jobs SET monthly_hits = '***Sep-2026=638' WHERE job_id= '1726985'`
0.0008	`SELECT date_sent FROM jobseeker_sent_emails WHERE jobseeker_id = '' AND job_id = '1726985' AND status LIKE 'sent%' ORDER BY id DESC`
0.0003	SELECT * FROM `employer_jobs_skills` `ejs` LEFT JOIN `skills_categories` `sc` ON `ejs`.`skill_id` = `sc`.`id` WHERE `job_id` = 1726985
0.0037	SELECT COUNT(*) AS `numrows` FROM `employer_jobs` WHERE `employer_id` = '884547' AND `date_added` >= '2022-06-08'
0.0004	`select * from teasers`
0.0002	`SELECT * FROM skill_categories WHERE skill_cat_id=''`

HTTP HEADERS (Show)

SESSION DATA (Show)

__ci_last_regenerate	1790480566
last_page	https://v1.stage.onlinejobs.ph/jobseekers/job/Senior-Web-Scraping-Architect-Python-Data-Engineer-Master-Level-Only-1726985
csrf-token	b40cb71ccca1a89d429daed362416e46

CONFIG VARIABLES (Show)

base_url	https://v1.stage.onlinejobs.ph
log_threshold	2
enable_profiler
index_page
uri_protocol	PATH_INFO
url_suffix
language	english
charset	UTF-8
enable_hooks	1
subclass_prefix	MY_
composer_autoload	vendor/autoload.php
permitted_uri_chars	a-z 0-9~%.:_\=\-\+\@?\&
enable_query_strings
controller_trigger	c
function_trigger	m
directory_trigger	d
allow_get_array	1
log_path
log_file_extension
log_file_permissions	420
log_date_format	Y-m-d H:i:s
error_views_path
cache_path
cache_query_string
encryption_key	OdBUArjiWg9I7u7bvAwQ7Fu35VB1kzga
sess_driver	files
sess_cookie_name	ci_session
sess_expiration	2678400
sess_save_path
sess_match_ip
sess_time_to_update	300
sess_regenerate_destroy
cookie_prefix
cookie_domain
cookie_path	/
cookie_secure
cookie_httponly
standardize_newlines
global_xss_filtering	1
csrf_protection
csrf_token_name	csrf_test_name
csrf_cookie_name	csrf_cookie_name
csrf_expire	7200
csrf_regenerate	1
csrf_exclude_uris	Array ( )
compress_output
time_reference	local
rewrite_short_tags
proxy_ips
ojadmin	JpjZkQN5A8l@^L
salt	+UFjSAT49tPZLtmU2CIG2FYN7pRhgsWyLHgSyQa6k3I=
queue_enabled
twilio	Array ( [sid] => AC00c0594045c6eef9407e8fff01f3d467 [token] => f0bfc0b73444a077894a43f5f75e6d41 [length] => 6 [from] => +639221200200 [code_expiry_seconds] => 14400 )
maintenance	Array ( [admin] => Array ( [verification] => ) [announcement_bar] => Array ( [show_slow_issue_message] => ) )
trustpilot	Array ( [to_email] => onlinejobs.ph+3d844ebf71@invite.trustpilot.com )
services	Array ( [google_tag_manager] => Array ( [id] => GTM-T5CQMS6P ) [chatgpt] => Array ( [secret] => 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 ) [claude] => Array ( [secret] => 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 ) )
v2_url	https://v2.onlinejobs.ph
replacemyself_secret_key	Kk1WpgTMkc4wBQOqC5rqCssdLhACKrsJeTtO1ywUkT4=
app	Array ( [app] => Array ( [command] => Array ( [map] => Array ( [OnlineJobs\Mailer\Command\SendEmailCommand] => OnlineJobs\Mailer\Handler\SendEmailHandler [OnlineJobs\Mail\Command\SendEmailCommand] => OnlineJobs\Mail\Handler\SendEmailHandler [OnlineJobs\Mail\Command\QueueEmailPendingCommand] => OnlineJobs\Mail\Handler\QueueEmailPendingHandler [OnlineJobs\Mail\Command\QueueJobEmailPendingCommand] => OnlineJobs\Mail\Handler\QueueJobEmailPendingHandler [OnlineJobs\Mail\Command\QueueUnapprovedEmailCommand] => OnlineJobs\Mail\Handler\QueueUnapprovedEmailHandler [OnlineJobs\Mail\Command\ProcessEmailQueueCommand] => OnlineJobs\Mail\Handler\ProcessEmailQueueHandler [OnlineJobs\Mail\Command\PutJobseekerEmailOnHoldCommand] => OnlineJobs\Mail\Handler\PutJobseekerEmailOnHoldHandler [OnlineJobs\Purchase\Command\RealEstateVaCourseCommand] => OnlineJobs\Purchase\Handler\RealEstateVaCourseHandler [OnlineJobs\Test\Command\HelloCommand] => OnlineJobs\Test\Handler\HelloHandler ) [middleware] => Array ( ) ) [providers] => Array ( [0] => OnlineJobs\Bus\BusServiceProvider [1] => OnlineJobs\Database\DbServiceProvider [2] => OnlineJobs\Mailer\MailerServiceProvider [3] => OnlineJobs\Support\SupportServiceProvider [4] => OnlineJobs\Employer\ServiceProvider [5] => OnlineJobs\Jobseeker\ServiceProvider [6] => OnlineJobs\Mail\ServiceProvider [7] => OnlineJobs\Postal\ServiceProvider [8] => OnlineJobs\Queue\ServiceProvider [9] => OnlineJobs\Courier\ServiceProvider [10] => OnlineJobs\Job\ServiceProvider [11] => OnlineJobs\Purchase\ServiceProvider ) ) )
meta	Array ( [facebook] => Array ( [input] => 1 [connect] => 2 [timezone] => 3 [url] => 30 [id] => 31 [photo] => 32 [name] => 33 [email] => 34 [review] => 35 ) [trust] => Array ( [government_id] => 4 [utility_bill] => 5 [selfie_photo] => 6 [profile_picture] => 7 [phone_number] => 8 [facebook] => 9 ) [verify] => Array ( [profile_picture] => 10 [government_id] => 11 [selfie_photo] => 12 [utility_bill] => 13 [address] => 14 [name] => 15 [reviewed] => 16 [government_group] => 28 [address_group] => 29 [mobile] => 38 ) [address] => Array ( [complete] => 17 [room_unit_no] => 20 [house_no] => 21 [street] => 22 [subdivision] => 23 [barangay] => 24 [city] => 25 [province] => 26 [postcode] => 27 ) [reviewed] => Array ( [similar_names] => 1 ) [name] => 18 [phone] => 19 [sms_verification_code] => 36 [resend_verification_code] => 37 [address_read] => 39 [first_name] => 100 [middle_name] => 101 [last_name] => 102 [edit] => Array ( [government_id_group] => 40 [address_group] => 41 [mobile_group] => 42 ) [salary_re_entry] => 103 [id_proof_recalibrate] => 104 [email] => 44 [mobile_verification_type] => 45 )
honeypot	Array ( [name_field] => my_name [enabled] => 1 )

HTTP_ACCEPT	/
HTTP_USER_AGENT	Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko; compatible; ClaudeBot/1.0; +claudebot@anthropic.com)
HTTP_CONNECTION	Upgrade
SERVER_PORT	80
SERVER_NAME	v1.stage.onlinejobs.ph
REMOTE_ADDR	127.0.0.1
SERVER_SOFTWARE	Apache/2.4.58 (Ubuntu)
HTTP_ACCEPT_LANGUAGE
SCRIPT_NAME	/index.php
REQUEST_METHOD	GET
HTTP_HOST
REMOTE_HOST
CONTENT_TYPE
SERVER_PROTOCOL	HTTP/1.1
QUERY_STRING
HTTP_ACCEPT_ENCODING	gzip, br, zstd, deflate
HTTP_X_FORWARDED_FOR	216.73.216.74
HTTP_DNT

Senior Web Scraping Architect & Python Data Engineer (Master-Level Only)

Please login or register as jobseeker to apply for this job.

TYPE OF WORK

SALARY

HOURS PER WEEK

DATE UPDATED

Why is this blurred?