NeGD is currently inviting applications for the Data Engineering Lead position on a contractual basis, initially for a period of three years, with possible extensions based on project requirements.

Responsibilities

Data Acquisition & Integration
• Build batch, streaming and change-data-capture ingestion from departmental databases, APIs and files, and connectors to national platforms
• Define the technical terms of data-sharing agreements with departments

Document & Unstructured Data Pipelines
• Build high-volume intake for scanned documents and images, with parsing and OCR pipelines run with the NLP & Document AI team, metadata capture and storage

Data Platform
• Run the lakehouse or warehouse, orchestration, transformation, quality checks, lineage and data catalogue on IndiaAI and on-premise infrastructure

Datasets for AI
• Maintain a registry of training data and golden evaluation datasets, with versioning, dataset documentation and access control
• Run annotation pipelines, including for Indian-language data, with the QA team

Retrieval Infrastructure & AI Kosh
• Build embedding pipelines and manage the lifecycle of vector indexes, including separate indexes per department
• Synchronise with the AI Kosh catalogue, and prepare datasets for publication on AIKosh

Data Governance & Privacy
• Apply classification, masking, anonymisation and retention of personal data under DPDPA 2023, and maintain access logs

Leadership & Standards
• Build pipelines directly, set data standards for vendor teams, and review their pipelines before acceptance

Important Links

About National e-Governance Division (NeGD)

The National e-Governance Division (NeGD) is an independent business division under the Digital India Corporation, Ministry of Electronics and Information Technology. NeGD has been playing a pivotal role in supporting MeitY in Programme Management and implementation of e-Governance projects and initiatives undertaken by various Ministries/ Departments, both at the Central and State levels.

NeGD has been spearheading several innovative initiatives under the aegis of the Digital India Programme. Those have been developed keeping the vision areas of Digital India at the core- providing digital infrastructure as a core utility to every citizen, governance and services on demand and in particular, digital empowerment of the citizens of our country; some of these initiatives include DigiLocker, UMANG, Poshan Tracker, OpenForge Platform, API Setu, National Academic Depository, Academic Bank of Credits, Learning Management System.