Databricks Data Engineer – Data Contracts, OCDS
Scicom Infrastructure Services
Posted: 10/21/2026AirflowApacheAWSAzureCloudETLJenkinsKafkaPySpark+5 more
Senior Software Engineer, Data Infrastructure – RDBMS
TRM Labs
Posted: 10/21/2026PostgresPythonSQLGo
Senior/Staff Backend Engineer, Platform & Data Ingestion
Verana Health
Posted: 10/21/2026AWSBigQueryCloudElasticSearchElixirETLGoogle Cloud PlatformJavaKafka+4 more
Data Engineer Internship
American Family Insurance
Posted: 10/21/2026AWSCloudGoogle Cloud PlatformPythonSQL
Controls Engineer – Data Centers
Barry-Wehmiller
Posted: 10/21/2026
Financial Management Data Engineer
Guidehouse
Posted: 10/21/2026AWSAzureCloudPySparkPythonSQL
Manager, Data Engineering
Capital Blue Cross
Posted: 10/21/2026CloudSDLC
Senior Data Engineer, Payments
Airbnb
Posted: 10/21/2026AirflowETLJavaPythonScalaSparkSQL
Data Engineer I, Databricks, MLOps
Travelers
Posted: 10/21/2026CloudTerraform
Manager, Data Engineering
Association of American Medical Colleges (AAMC)
Posted: 10/21/2026AWSCloudETLOraclePostgresSDLC
Senior Data Engineer
The Ohio State University, Main Campus
Posted: 10/21/2026ETL
Senior Data Engineer
Thomson Reuters
Posted: 10/21/2026ETLPythonSQLTableau
Software Engineer, Data Platform Lead – Horizon AI
STN Incorporated
Posted: 10/21/2026Postgres
Data Engineer
General Dynamics Information Technology
Posted: 10/21/2026ApacheCloudETLPySparkPythonSparkSQL
Staff Software Engineer – Data Acquisition
WEX
Posted: 10/21/2026Distributed SystemsJavaPythonScala
Principal Data Architect
Public Partnerships | PPL
Posted: 10/21/2026ApacheAzureCloudPythonSparkSQLUnity
Data Engineer/Architect
Leidos
Posted: 10/21/2026ETLJavaScriptPostgresPython
Principal Engineer – Data Science
Verizon
Posted: 10/21/2026CloudGrafanaHadoopPythonSparkSplunkSQLTableau
Principal Engineer – Data Science
Verizon
Posted: 10/21/2026CloudGrafanaHadoopPythonSparkSplunkSQLTableau
Lead Cloud Data Engineer
Baptist Health
Posted: 10/21/2026AirflowAmazon RedshiftApacheAWSBigQueryCloudDistributed SystemsDockerGoogle Cloud Platform+4 more
Databricks Data Engineer – Data Contracts, OCDS
Scicom Infrastructure ServicesCore Competencies
Role fitCore Competencies
Use this summary to align your resume positioning with the role.
Demonstrates expertise in implementing the Open Contracting Data Standard (OCDS) and managing data contracts while leveraging Databricks for data engineering and analytics. Proficient in translating business requirements into technical specifications and ensuring data quality and compliance across procurement processes.
Highest-signal resume keywords
Open Contracting Data Standard (OCDS)Databricks Data EngineeringApache SparkData Contract ManagementData Quality Validation
Tech Stack
Tools & technologiesAbout the role
Key responsibilities & impact- Bridge business, contracting, data-governance, and engineering teams for a large-scale Databricks implementation
- Lead requirements-gathering sessions with procurement, contracting, program, analytics, governance, and technical stakeholders
- Define, construct, document, and maintain data contracts between data producers and consumers
- Translate business requirements into user stories, acceptance criteria, process flows, data mappings, interface specifications, and technical requirements
- Conduct source-system analysis, data profiling, gap assessments, and source-to-target mapping
- Identify differences between existing procurement data and required OCDS structures
- Facilitate agreement among data owners, producers, consumers, architects, and governance teams
- Maintain traceability through data models, engineering implementation, testing, and acceptance
- Evaluate requested changes for downstream effects on data products, reports, integrations, and analytical use cases
- Support backlog refinement, sprint planning, demonstrations, testing, and stakeholder acceptance
- Apply OCDS to procurement and public-contracting datasets across planning, tender, award, contract, and implementation stages
- Develop and maintain mappings for OCDS releases, records, packages, identifiers, organizations, parties, items, milestones, documents, transactions, amendments, and related elements
- Interpret and apply OCDS schemas, codelists, validation rules, and implementation guidance
- Determine when OCDS extensions are necessary and document assumptions, business rules, mappings, extensions, and exceptions
- Construct contracting records from multiple transactional releases and establish rules for amendments, updates, cancellations, corrections, and historical changes
- Validate transformed data against OCDS JSON schemas and resolve missing, invalid, inconsistent, or nonconforming procurement data
- Support publication, exchange, analysis, or internal use of standardized contracting data
- Design, develop, test, and maintain Databricks data pipelines using Apache Spark, PySpark, Python, and SQL
- Build ingestion and transformation pipelines for structured and semi-structured procurement data from JSON, CSV, XML, Parquet, relational databases, APIs, and files
- Implement Bronze, Silver, and Gold data layers using medallion architecture
- Build normalized, dimensional, analytical, and OCDS-aligned data models
- Use Delta Lake for schema enforcement, schema evolution, versioning, auditability, and reliable processing
- Develop reusable frameworks for mapping source procurement data into OCDS-compatible outputs
- Implement batch and incremental ingestion patterns
- Use Databricks Workflows, notebooks, jobs, Auto Loader, Delta Live Tables, or Lakeflow as appropriate
- Develop REST API integrations for source ingestion and downstream data delivery
- Implement automated data-quality, reconciliation, completeness, and conformity checks
- Support Unity Catalog implementation for metadata, ownership, lineage, access control, and data discovery
- Optimize Spark jobs, SQL queries, clusters, partitioning, file sizes, and data layouts
- Participate in code reviews, automated testing, CI/CD, deployment, monitoring, and production support
- Investigate pipeline failures, data discrepancies, and performance issues
- Define measurable data-quality rules for accuracy, completeness, validity, timeliness, consistency, and uniqueness
- Develop validation controls for OCDS fields, identifiers, dates, amounts, currencies, organizations, classifications, and contracting relationships
- Create dashboards or reports showing data-contract compliance and data-quality results
- Establish processes for detecting and managing schema drift
- Document data lineage from procurement systems through Databricks transformations and downstream products
- Work with governance teams to assign data owners, stewards, classifications, retention requirements, and access policies
- Ensure sensitive procurement and supplier information is handled according to security and privacy requirements
- Support auditability through documented rules, version-controlled mappings, validation results, and change histories
Requirements
What you’ll need- Bachelor’s degree in computer science, information systems, data analytics, business analysis, engineering, public administration, supply-chain management, or a related field
- At least five years of combined data-engineering, data-analysis, business-analysis, or data-integration experience
- Hands-on experience implementing or working with the Open Contracting Data Standard (OCDS)
- Demonstrated experience constructing, documenting, negotiating, or maintaining data contracts
- Experience mapping procurement or contracting data to OCDS schemas
- Strong knowledge of OCDS releases, records, schemas, codelists, identifiers, contracting stages, and validation practices
- At least three years of hands-on experience with Databricks
- Strong experience with Apache Spark, PySpark, Python, SQL, Delta Lake, ETL and ELT pipelines, data modeling, JSON and JSON Schema, REST APIs, data-quality validation, and source-to-target mapping
- Experience gathering requirements and translating them into implementable engineering specifications
- Ability to communicate effectively with technical and nontechnical stakeholders
- Experience writing user stories, acceptance criteria, business rules, data dictionaries, interface specifications, and process documentation
- Strong analytical, troubleshooting, facilitation, and documentation skills
- Experience working within Agile delivery teams
- Preferred: experience with government procurement, public-sector contracting, grants, acquisition, supplier, or financial data
- Preferred: experience implementing OCDS extensions or tailoring OCDS for organizational requirements
- Preferred: familiarity with procurement classifications, organizational identifiers, tender processes, awards, amendments, milestones, transactions, and contract implementation
- Preferred: Databricks Certified Data Engineer Associate or Professional certification
- Preferred: experience with Unity Catalog, Delta Live Tables, Lakeflow, Databricks Workflows, Structured Streaming, cloud platforms, Azure Data Factory, ADLS Gen2, AWS Glue, Amazon S3, Snowflake, dbt, Kafka, Airflow, Power BI, JSON Schema validation tools, automated data-quality frameworks, metadata management, master-data management, data lineage, reference-data governance, CI/CD, Git, Azure DevOps, GitHub Actions, Jenkins, or Terraform
- Preferred: experience supporting large federal, state, local-government, or regulated-enterprise data programs
- Preferred: familiarity with federal acquisition, procurement, reporting, transparency, or open-data requirements
- Ability to operate in technical engineering and business-analysis discussions
- Strong understanding of data contracts and accountability between data producers and consumers
- Ability to convert complex procurement processes into clear data structures and transformation rules
- Attention to detail when interpreting schemas, codelists, business definitions, and validation requirements
- Strong stakeholder-facilitation and conflict-resolution skills
- Ability to identify gaps and ambiguities before they become engineering defects
- Commitment to documentation, traceability, governance, and data quality
- Ability to work effectively within a large, multidisciplinary Databricks team
Benefits
Comp & perks- No benefits, perks, or compensation extras are specified in the posting.
ATS Keywords
✓ Tailor your resumeApplicant Tracking System Keywords
Tip: use these terms in your resume and cover letter to boost ATS matches.
Hard Skills
Data MappingETL and ELT PipelinesSQLPythonPySparkDelta LakeJSON SchemaData ModelingREST APIsSource-to-Target Mapping
Soft Skills
Analytical SkillsFacilitation SkillsTroubleshooting SkillsCommunication SkillsAttention to Detail
Tools & Technologies
Databricks WorkflowsDelta Live TablesUnity CatalogAzure Data FactoryAWS GlueSnowflakePower BIGitJenkinsTerraform
Certifications & Qualifications
Databricks Certified Data Engineer AssociateDatabricks Certified Data Engineer Professional
Industry Keywords
Government ProcurementPublic-Sector ContractingData GovernanceData LineageOpen Data Requirements