Legal AI Product Tester / Legal QA Analyst

Please login or register as jobseeker to apply for this job.

: TYPE OF WORK

Part Time

: SALARY

4.50 per hour

: HOURS PER WEEK

30

: DATE UPDATED

May 29, 2026

JOB OVERVIEW

Legal AI Product Tester / Legal QA Analyst

Remote | Contract | Philippines-Based Preferred

About the Role

Irys is looking for lawyers or legally trained professionals to test our legal AI platform on a daily basis. This role is focused on stress-testing legal AI workflows across litigation, corporate, transactional, research, drafting, and document-review use cases.

You will use your own legal judgment, test sets, sample documents, pleadings, contracts, memos, and research tasks to evaluate whether the system produces accurate, useful, well-structured, and lawyer-reviewable work product.

This is not a passive QA role. We need someone who can think like a lawyer, identify what went wrong, explain why it matters, and structure feedback clearly for our product and AI teams.

Responsibilities

Test Irys legal AI workflows daily across different legal tasks.
Bring and create realistic test sets, including pleadings, contracts, memos, client emails, research questions, diligence materials, and mixed document bundles.
Evaluate outputs for legal accuracy, factual grounding, completeness, structure, tone, formatting, and usefulness.
Identify hallucinations, missed facts, weak reasoning, unsupported claims, bad drafting choices, citation problems, and confusing product behavior.
Compare outputs across different modes, prompts, models, and workflows.
Provide structured feedback with clear issue descriptions, severity levels, screenshots, source references, and recommended fixes.
Work closely with product, engineering, and AI teams to reproduce issues and improve the system.
Help define testing rubrics and repeatable evaluation sets for future benchmarking.

What You Should Be Testing For

Did the system understand the uploaded documents?
Did it extract the right facts and miss anything important?
Did it distinguish between fact, law, argument, risk, and client narrative?
Did it cite or reference sources correctly?
Did it draft in the right legal format?
Did it over-admit, overstate, hallucinate, or make unsupported conclusions?
Did it produce something a lawyer could actually review and use?
Did the workflow make sense from a user perspective?
Did any internal/debug language appear in the final output?
Did the product behave reliably across repeated tests?

Qualifications

Law degree or substantial legal training required.
Strong familiarity with U.S. legal documents, pleadings, motions, contracts, legal memos, and common law reasoning strongly preferred.
Experience with U.S., U.K., Australian, Singaporean, or other common law legal work is a plus.
Litigation and/or corporate/transactional experience preferred.
Strong legal writing and analytical skills.
Excellent attention to detail.
Comfortable testing software, documenting bugs, and giving precise feedback.
Ability to work independently and test consistently without needing constant direction.
Interest in legal AI, legal technology, and product development.

Ideal Candidate

The ideal candidate is a legally trained operator who can test like a lawyer and think like a product reviewer. You should be comfortable saying:

This answer is legally wrong.
This draft is useful but not filing-ready.
This missed a key fact.
This cited the wrong source.
This workflow is confusing.
This is a serious issue versus a minor polish issue.

Engagement

Remote contract role.
Part-time or full-time depending on fit and availability.
Daily testing expected.
Paid trial period may be required.
Must be able to provide structured written feedback in clear English.

To Apply

Please send:

Resume or LinkedIn profile.
Short summary of your legal background.
Jurisdictions or legal systems you are familiar with.
Examples of legal documents or workflows you would be comfortable testing.
Availability and expected hourly or monthly rate.
Any prior experience with AI tools, legal tech, QA, or product testing.

SKILL REQUIREMENT

Legal Services Usability Testing AI Tools

SHARE THIS POST

Job Post	Reporter	Rank	Reason	Date

BENCHMARKS

Loading Time: Base Classes	0.0007
Controller Execution Time ( Jobseekers / Job )	0.0133
Total Execution Time	0.0147

GET DATA

No GET data exists

MEMORY USAGE

1,502,688 bytes

POST DATA

No POST data exists

URI STRING

jobseekers/job/Legal-AI-Product-Tester-Legal-QA-Analyst-1657506

CLASS/METHOD

jobseekers/job

DATABASE: onlinejobs (Jobseekers:$db) QUERIES: 13 (0.0079 seconds) (Hide)

0.0003	`SELECT * FROM exrates WHERE rate_name = 'USD-PHP'`
0.0004	SELECT * FROM `employer_jobs` WHERE `job_id` = 1657506 LIMIT 1
0.0011	SELECT * FROM `employers` WHERE `employer_id` = 939601 LIMIT 1
0.0009	SELECT COUNT(*) AS `numrows` FROM `t_thread` `t` LEFT JOIN `t_thread_misc` `misc` ON `t`.`id` = `misc`.`thread_id` WHERE `t`.`job_id` = 1657506 AND `misc`.`id` IS NULL
0.0005	SELECT e.business_name, e.logo, e.website, e.rebill_date, e.date_added member_date, hits, DATEDIFF('2026-07-11',ej.date_added) duration_days, DATEDIFF('2026-07-11',e.rebill_date) duration_rebill, ej.*, e.deactivate FROM employers e, employer_jobs ej WHERE e.employer_id = ej.employer_id AND ((e.user_level >= '500' AND ej.date_added <= e.rebill_date) OR e.employer_id = '' OR (ej.date_approved <> '2000-01-01' and DATEDIFF('2026-07-11',ej.date_added) <= 14 )) AND e.deactivate != 1 AND ej.deleted = 0 AND job_id = '1657506'
0.0008	SELECT * FROM `employer_jobs_skills` `ejs` LEFT JOIN `skills_categories` `sc` ON `ejs`.`skill_id` = `sc`.`id` WHERE `job_id` = 1657506
0.0007	UPDATE employer_jobs SET hit_counts = '*May-29-2026=600May-30-2026=63May-31-2026=42Jun-01-2026=47Jun-02-2026=45Jun-03-2026=27Jun-04-2026=16Jun-05-2026=22Jun-06-2026=18Jun-07-2026=16Jun-08-2026=23Jun-09-2026=14Jun-10-2026=6Jun-11-2026=13Jun-12-2026=5Jun-13-2026=8Jun-14-2026=11Jun-15-2026=18Jun-16-2026=19Jun-17-2026=15Jun-18-2026=17Jun-19-2026=13Jun-20-2026=14Jun-21-2026=7Jun-22-2026=11Jun-23-2026=18Jun-24-2026=11Jun-25-2026=10Jun-26-2026=8Jun-27-2026=5Jun-28-2026=7Jun-29-2026=13Jun-30-2026=14Jul-01-2026=14Jul-02-2026=4Jul-03-2026=4Jul-04-2026=2*Jul-11-2026=1' WHERE job_id= '1657506'
0.0006	`UPDATE employer_jobs SET monthly_hits = '*May-2026=705Jun-2026=471**Jul-2026=25' WHERE job_id= '1657506'`
0.0009	`SELECT date_sent FROM jobseeker_sent_emails WHERE jobseeker_id = '' AND job_id = '1657506' AND status LIKE 'sent%' ORDER BY id DESC`
0.0003	SELECT * FROM `employer_jobs_skills` `ejs` LEFT JOIN `skills_categories` `sc` ON `ejs`.`skill_id` = `sc`.`id` WHERE `job_id` = 1657506
0.0008	SELECT COUNT(*) AS `numrows` FROM `employer_jobs` WHERE `employer_id` = '939601' AND `date_added` >= '2022-06-08'
0.0004	`select * from teasers`
0.0002	`SELECT * FROM skill_categories WHERE skill_cat_id=''`

HTTP HEADERS (Show)

SESSION DATA (Show)

__ci_last_regenerate	1783733757
last_page	https://v1.stage.onlinejobs.ph/jobseekers/job/Legal-AI-Product-Tester-Legal-QA-Analyst-1657506
csrf-token	2331bce08644bf4fa93d79dfd31d19f7
job_search_keyword

CONFIG VARIABLES (Show)

base_url	https://v1.stage.onlinejobs.ph
log_threshold	2
enable_profiler
index_page
uri_protocol	PATH_INFO
url_suffix
language	english
charset	UTF-8
enable_hooks	1
subclass_prefix	MY_
composer_autoload	vendor/autoload.php
permitted_uri_chars	a-z 0-9~%.:_\=\-\+\@?\&
enable_query_strings
controller_trigger	c
function_trigger	m
directory_trigger	d
allow_get_array	1
log_path
log_file_extension
log_file_permissions	420
log_date_format	Y-m-d H:i:s
error_views_path
cache_path
cache_query_string
encryption_key	OdBUArjiWg9I7u7bvAwQ7Fu35VB1kzga
sess_driver	files
sess_cookie_name	ci_session
sess_expiration	2678400
sess_save_path
sess_match_ip
sess_time_to_update	300
sess_regenerate_destroy
cookie_prefix
cookie_domain
cookie_path	/
cookie_secure
cookie_httponly
standardize_newlines
global_xss_filtering	1
csrf_protection
csrf_token_name	csrf_test_name
csrf_cookie_name	csrf_cookie_name
csrf_expire	7200
csrf_regenerate	1
csrf_exclude_uris	Array ( )
compress_output
time_reference	local
rewrite_short_tags
proxy_ips
ojadmin	JpjZkQN5A8l@^L
salt	+UFjSAT49tPZLtmU2CIG2FYN7pRhgsWyLHgSyQa6k3I=
queue_enabled
twilio	Array ( [sid] => AC00c0594045c6eef9407e8fff01f3d467 [token] => f0bfc0b73444a077894a43f5f75e6d41 [length] => 6 [from] => +639221200200 [code_expiry_seconds] => 14400 )
maintenance	Array ( [admin] => Array ( [verification] => ) [announcement_bar] => Array ( [show_slow_issue_message] => ) )
trustpilot	Array ( [to_email] => onlinejobs.ph+3d844ebf71@invite.trustpilot.com )
services	Array ( [google_tag_manager] => Array ( [id] => GTM-T5CQMS6P ) [chatgpt] => Array ( [secret] => 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 ) [claude] => Array ( [secret] => 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 ) )
v2_url	https://v2.onlinejobs.ph
replacemyself_secret_key	Kk1WpgTMkc4wBQOqC5rqCssdLhACKrsJeTtO1ywUkT4=
app	Array ( [app] => Array ( [command] => Array ( [map] => Array ( [OnlineJobs\Mailer\Command\SendEmailCommand] => OnlineJobs\Mailer\Handler\SendEmailHandler [OnlineJobs\Mail\Command\SendEmailCommand] => OnlineJobs\Mail\Handler\SendEmailHandler [OnlineJobs\Mail\Command\QueueEmailPendingCommand] => OnlineJobs\Mail\Handler\QueueEmailPendingHandler [OnlineJobs\Mail\Command\QueueJobEmailPendingCommand] => OnlineJobs\Mail\Handler\QueueJobEmailPendingHandler [OnlineJobs\Mail\Command\QueueUnapprovedEmailCommand] => OnlineJobs\Mail\Handler\QueueUnapprovedEmailHandler [OnlineJobs\Mail\Command\ProcessEmailQueueCommand] => OnlineJobs\Mail\Handler\ProcessEmailQueueHandler [OnlineJobs\Mail\Command\PutJobseekerEmailOnHoldCommand] => OnlineJobs\Mail\Handler\PutJobseekerEmailOnHoldHandler [OnlineJobs\Purchase\Command\RealEstateVaCourseCommand] => OnlineJobs\Purchase\Handler\RealEstateVaCourseHandler [OnlineJobs\Test\Command\HelloCommand] => OnlineJobs\Test\Handler\HelloHandler ) [middleware] => Array ( ) ) [providers] => Array ( [0] => OnlineJobs\Bus\BusServiceProvider [1] => OnlineJobs\Database\DbServiceProvider [2] => OnlineJobs\Mailer\MailerServiceProvider [3] => OnlineJobs\Support\SupportServiceProvider [4] => OnlineJobs\Employer\ServiceProvider [5] => OnlineJobs\Jobseeker\ServiceProvider [6] => OnlineJobs\Mail\ServiceProvider [7] => OnlineJobs\Postal\ServiceProvider [8] => OnlineJobs\Queue\ServiceProvider [9] => OnlineJobs\Courier\ServiceProvider [10] => OnlineJobs\Job\ServiceProvider [11] => OnlineJobs\Purchase\ServiceProvider ) ) )
meta	Array ( [facebook] => Array ( [input] => 1 [connect] => 2 [timezone] => 3 [url] => 30 [id] => 31 [photo] => 32 [name] => 33 [email] => 34 [review] => 35 ) [trust] => Array ( [government_id] => 4 [utility_bill] => 5 [selfie_photo] => 6 [profile_picture] => 7 [phone_number] => 8 [facebook] => 9 ) [verify] => Array ( [profile_picture] => 10 [government_id] => 11 [selfie_photo] => 12 [utility_bill] => 13 [address] => 14 [name] => 15 [reviewed] => 16 [government_group] => 28 [address_group] => 29 [mobile] => 38 ) [address] => Array ( [complete] => 17 [room_unit_no] => 20 [house_no] => 21 [street] => 22 [subdivision] => 23 [barangay] => 24 [city] => 25 [province] => 26 [postcode] => 27 ) [reviewed] => Array ( [similar_names] => 1 ) [name] => 18 [phone] => 19 [sms_verification_code] => 36 [resend_verification_code] => 37 [address_read] => 39 [first_name] => 100 [middle_name] => 101 [last_name] => 102 [edit] => Array ( [government_id_group] => 40 [address_group] => 41 [mobile_group] => 42 ) [salary_re_entry] => 103 [id_proof_recalibrate] => 104 [email] => 44 [mobile_verification_type] => 45 )
honeypot	Array ( [name_field] => my_name [enabled] => 1 )

HTTP_ACCEPT	/
HTTP_USER_AGENT	Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko; compatible; ClaudeBot/1.0; +claudebot@anthropic.com)
HTTP_CONNECTION	Upgrade
SERVER_PORT	80
SERVER_NAME	v1.stage.onlinejobs.ph
REMOTE_ADDR	127.0.0.1
SERVER_SOFTWARE	Apache/2.4.58 (Ubuntu)
HTTP_ACCEPT_LANGUAGE
SCRIPT_NAME	/index.php
REQUEST_METHOD	GET
HTTP_HOST
REMOTE_HOST
CONTENT_TYPE
SERVER_PROTOCOL	HTTP/1.1
QUERY_STRING
HTTP_ACCEPT_ENCODING	gzip, br, zstd, deflate
HTTP_X_FORWARDED_FOR	216.73.217.20
HTTP_DNT

Legal AI Product Tester / Legal QA Analyst

Please login or register as jobseeker to apply for this job.

TYPE OF WORK

SALARY

HOURS PER WEEK

DATE UPDATED

Why is this blurred?