SUK · Third Year B.Tech. Computer Science and Engineering (Data Science) · NEP 2020
Data Acquisition
DS3205 · Elective III · Semester VI · 3 credits · ISE 10 + mid-semester 30 + ESE 60
One of the Elective III options.
Unit 1: Introduction to Data Acquisition
7 hoursData acquisition in the data-science life cycle; structured, semi-structured and unstructured data; internal and external sources; CSV, JSON, XML, Parquet and Avro; collection challenges and quality issues; ethics, law and privacy (GDPR, consent).
Examined in the 60-mark end-semester paper, and in the mid-semester exam if it is taught by then
- Data science lifecycle: CRISP-DM
- Types of digital data: structured, semi-structured and unstructured
- Data serialisation formats
- Data collection strategies
- Data privacy regulations: GDPR, HIPAA and CCPA
Unit 2: Data Acquisition from Files and Databases
8 hoursText, CSV, Excel and log files; SQL extraction with joins, subqueries and views; key-value, document, column-family and graph databases; ingestion with Pandas and SQLAlchemy.
Examined in the 60-mark end-semester paper, and in the mid-semester exam if it is taught by then
- Pandas data frames
- SQL joins, subqueries and CTEs
- Key-value and document stores
- Data ingestion: types, tools and challenges
Unit 3: Web-Based Data Acquisition
7 hoursWeb data sources and RESTful APIs over HTTP; public APIs returning JSON and XML; web scraping with BeautifulSoup and Scrapy, pagination, authentication and rate limits; legal and ethical issues.
Examined in the 60-mark end-semester paper, and in the mid-semester exam if it is taught by then
- Web services: SOAP and REST
- Ethics and privacy in data science
Unit 4: Sensor, Streaming and Real-Time Data Acquisition
7 hoursSensor and IoT data sources; acquisition from devices and embedded systems; streaming concepts, message brokers and platforms: Kafka, Flume and MQTT; real-time ingestion pipelines and their use cases.
Examined in the 60-mark end-semester paper, and in the mid-semester exam if it is taught by then
- IoT protocols: MQTT, CoAP and AMQP
- Event-driven architecture and publish-subscribe
- Apache Kafka architecture
- Data ingestion: Flume and Sqoop
- Stream processing and windowing
Unit 5: Big Data and Cloud-Based Data Acquisition
7 hoursAcquisition in big-data ecosystems and HDFS ingestion; Sqoop and NiFi; cloud sources on AWS (S3, Kinesis), Google Cloud (BigQuery, Pub/Sub) and Azure (Data Factory, Event Hubs); data versioning and metadata management.
Examined in the 60-mark end-semester paper, and in the mid-semester exam if it is taught by then
- HDFS architecture
- Data pipeline architecture
- Cloud big data platforms: AWS and Google Cloud
Unit 6: Data Quality, Pre-processing and Governance
6 hoursValidation and cleansing during acquisition; missing, noisy and inconsistent data; provenance and lineage; governance frameworks; security in acquisition pipelines; case studies in healthcare, finance and social media.
Examined in the 60-mark end-semester paper, and in the mid-semester exam if it is taught by then
- Data quality and validation
- Handling missing data
- Data lineage and catalogues
- Data governance
Unit contents are summarised from the official SUK syllabus . AiCanCode.org is not affiliated with Shivaji University, Kolhapur.