Data Overview

What you'll see here

Baseline shape and data-quality checks: 25,480 cases, 11 features + target, moderate class imbalance, and three concrete cleanup requirements before modelling.

Data quality flags

no_of_employees has 0 rows with negative values

Fix: Clip to a minimum of 1 (employer must have ≥1 employee).

yr_of_estab has 0 rows before 1900

Fix: Keep as-is but engineer company_age = 2016 − yr_of_estab and cap at 0.

prevailing_wage mixes Hour / Week / Month / Year

Fix: Annualise: hourly ×2080, weekly ×52, monthly ×12; then log-transform.

0 missing values across all 0 columns

Fix: No imputation required.

Data structure — dtypes, non-nulls & cardinality

df.info() equivalent — 25,480 rows × 12 columns, 0 duplicate rows, 0 missing values in total.

Column	Dtype	Non-null	Unique
case_id	object	25,480	25,480
continent	object	25,480	6
education_of_employee	object	25,480	4
has_job_experience	object	25,480	2
requires_job_training	object	25,480	2
no_of_employees	int64	25,480	7,105
yr_of_estab	int64	25,480	199
region_of_employment	object	25,480	5
prevailing_wage	float64	25,480	25,454
unit_of_wage	object	25,480	4
full_time_position	object	25,480	2
case_status	object	25,480	2

Statistical summary — numeric variables

df.describe() on the raw + engineered numeric columns.

Variable	Count	Mean	Std	Min	25%	Median	75%	Max	Skew
no_of_employees	25,480	5,667	22,878	-26	1,022	2,109	3,504	602,069	12.27
yr_of_estab	25,480	1,979	42.37	1,800	1,976	1,997	2,005	2,016	-2.04
prevailing_wage	25,480	74,456	52,816	2.14	34,015	70,308	107,736	319,210	0.76

Statistical summary — categorical variables

Variable	Unique	Most frequent	Freq	% of rows
continent	6	Asia	16,861	66.2%
education_of_employee	4	Bachelor's	10,234	40.2%
has_job_experience	2	Y	14,802	58.1%
requires_job_training	2	N	22,525	88.4%
region_of_employment	5	Northeast	7,195	28.2%
unit_of_wage	4	Year	22,962	90.1%
full_time_position	2	Y	22,773	89.4%
case_status	2	Certified	17,018	66.8%

Data dictionary

Column	Type	Notes
case_id	id	Unique application id (dropped from modelling).
continent	categorical (6)	Continent the employee is from.
education_of_employee	categorical (4)	High School, Bachelor's, Master's, Doctorate.
has_job_experience	binary Y/N	Does the applicant have prior job experience?
requires_job_training	binary Y/N	Does the employer require training?
no_of_employees	numeric	Employer headcount. Contains negative outliers.
yr_of_estab	numeric	Year the employer was established. A few pre-1900 outliers.
region_of_employment	categorical (5)	West, Northeast, South, Midwest, Island.
prevailing_wage	numeric	Prevailing wage in the employment location.
unit_of_wage	categorical (4)	Hour / Week / Month / Year — must be normalised.
full_time_position	binary Y/N	Full-time or part-time role.
case_status	target	Certified / Denied — binary target.

Preview — first 8 rows

case_id	continent	education	exp	employees	yr	region	wage	unit	status