FREE ACCESS
5,000–10,000 jobs/day
See all jobs on Scoutfield
Search thousands of fresh jobs every day.
Discover
- Fresh listings
- Fast filters
- No subscription required
Create a free account and start exploring right away.
Core Competencies
Role fitCore Competencies
Use this summary to align your resume positioning with the role.
Proficient in developing and managing data pipelines using Microsoft Fabric and OneLake, with a strong focus on data quality, validation, and documentation. Demonstrates expertise in SQL, PySpark, and the medallion architecture across Bronze, Silver, and Gold layers.
Highest-signal resume keywords
Microsoft FabricOneLakeSQLPySparkData Pipeline Development
ATS Keywords
Tailor your resumeApplicant Tracking System Keywords
Tip: use these terms in your resume and cover letter to boost ATS matches.
Hard Skills
Data Pipeline ArchitectureIncremental LoadingChange DetectionSchema ManagementData QualityREST API IntegrationTechnical DocumentationTest ExecutionData ValidationVersion Control
Soft Skills
Analytical ThinkingAttention to DetailClear CommunicationCollaborationProactivity
Tools & Technologies
SharePoint OnlineNotebooksMedallion ArchitectureOperational DashboardsExecution Logging
Industry Keywords
Data EngineeringData QualityObservabilityContinuous ImprovementSustainability
Tech Stack
Tools & technologiesPySparkPythonSparkSQL
About the role
Key responsibilities & impact- Configure and test the connection to SharePoint Online for the defined pilot scope
- Develop pipeline components to ingest selected documents and metadata into the OneLake Bronze layer
- Implement incremental loading patterns, change detection, retries, execution logging, and exception handling
- Develop Silver layer logic for cleansing, normalization, deduplication, version handling, and conflict resolution
- Implement Gold layer components to prepare validated information products in accordance with business rules
- Support attribute extraction, information enrichment, and validation by business stakeholders
- Record lineage between source documents, pipeline executions, transformed records, and validated results
- Test document creation, update, versioning, duplication, deletion, exception, and modification scenarios
- Implement operational dashboards, alerts, execution history, logs, and support diagnostics
- Collaborate with SharePoint and information architecture teams to understand source structures, metadata, permissions, and change behavior
- Document technical constraints, unsupported behaviors, workarounds, performance, and production considerations
- Package code, notebooks, configurations, and pipeline definitions using approved version control and release practices
- Produce tests, technical documentation, deployment instructions, and operational runbooks
- Build and validate the ingestion flow for documents, metadata, and selected data from SharePoint Online into OneLake
- Work under the guidance of the Fabric and data architecture teams, in collaboration with SharePoint, information architecture, AI, and business teams
Requirements
What you’ll need- Hands-on experience with Microsoft Fabric, OneLake, lakehouses, notebooks, and data pipelines
- Knowledge of Spark-, PySpark-, or SQL-based transformations
- Experience with pipeline architecture and development, incremental loads, and change detection
- Knowledge of schema management, data quality, observability, recovery, and exception handling
- Experience integrating REST APIs, files, document metadata, or Microsoft 365 data sources
- Proficiency in SQL and at least one data engineering language, such as Python or PySpark
- Knowledge of medallion architecture and data organization across the Bronze, Silver, and Gold layers
- Experience with source control, parameterization, environment configuration, deployment, and testing
- Ability to develop repeatable tests that demonstrate the platform’s actual behavior
- Experience creating technical documentation, runbooks, deployment procedures, and support materials
- Intermediate to advanced English (B2 or higher), with the ability to communicate verbally and in writing in a corporate and technical environment
- Availability to work in the Eastern Standard Time (EST) time zone
- Analytical thinking and technical investigation skills
- Rigor in data validation, test execution, and results documentation
- Ability to troubleshoot and investigate integration or processing failures
- Organization, attention to detail, and engineering discipline
- Collaboration with architects, developers, and business professionals
- Clear communication regarding platform constraints, risks, failures, and behavior
- Proactivity and a focus on automation and continuous improvement
- Commitment to quality, traceability, observability, and sustainability
Benefits
Comp & perks- CLT employment option
- SulAmerica health insurance for you (nationwide coverage, ward accommodation, and copayments)
- TotalPass gym membership
- Latest-generation laptop
- Transportation allowance
- Meal allowance: BRL 770.00 (based on an average of 22 business days per month), with the option to allocate the amount to VR or a Flash card
- Candidate referral bonus
- Creditas financial services partnership
- Annual performance review with an IDP (Individual Development Plan)
- Training through iblue Academy
- Udemy training
- Certifications (AWS, Microsoft, IBM, and H2O)
- Educational partnerships (with possible financial assistance tied to your performance review)
- Structured Y-shaped career path (you can choose a management or specialist track)
- Benefit packages offered by the cooperative, which you can select according to your needs
- 15 days of paid leave after the 12th month