COMP9321 Data Services Engineering Term1, 2025 Week 1: Course Overview 2Teaching Team feel free to schedule consultations 8-9PM Every Monday. – Office: WFH /Consultations via Microsoft Teams ? Course Administrator
What was taught and why needed revision ? How to build Web sites using Java ? Standardised frameworks for Web apps (plenty) Many Web apps are now data-oriented or utilise data heavily
–functionality requires combining or processing complex data from multiple sources So COMP9321 became Data Service Engineering: ? How to work with data ? How to make the design and implementation of data-oriented service easy (i.e., an approach/technique) 4So what is this course about? Data Services Engineering Data = is the problem we want to deal with, understanding the problems and possible ways to
work with Data (e.g., “get” data, “publish” data, discover or manage multiple data sources, etc). Services = is the proposed solution/design approach to make our problem “manageable”.
Engineering = (best practices, weighing options, we will think about these all throughout, at least
try to) - obtain conceptual ideas as well as practical skills 5Course Aims This course aims to introduce the student to core concepts and practical skills for engineering the data in service-oriented
data-driven applications. Specifically, the course aims to answer these questions: ? How to access and ingest data from various external sources? ? How to process and store the data for applications? ? How to curate (e.g. Extract, Transform, Correct, Aggregate, and Merge/Split) and publish the data? ? How to visualize the data to communicate effectively ? How to apply available analytics to the data? Fundamentally, we will look at these questions through the lens of 'service-oriented' software design and implementation
principles. At each topic, we will learn some core concepts, and how to implement the concepts in software through
services. 6Assumed Knowledge Before commencing this course, we will assume that students have: ? completed one programming course (expected to be in Python) ? basic data modelling and relational database knowledge These are assumed to have been acquired in the following courses: For Postgrad - COMP9021
and COMP9311. For Undergrad - COMP1531 and COMP2041. NOTE: This course is not meant to be an advanced course … 7Course Structure Working with
Data ? Ingesting the data ? Cleaning and manipulating the data ? Visualizing the data Building a
Data service ? Building a RESTful API server ? Building a RESTful API client Data
Analytics ? Data Analytics Techniques and tools 8Assessments Assessment: ? 40% formal online exam: individual assessment. ? 50% on Individual assignment work – Assgn1 on Data ingestion, manipulation and visualization (individual) 15% – Assgn2 on building a service 20% – Assgn3 on building a data analytics service 15% ? 10% on 5 online quizzes (WebCMS-based quiz system, ‘open’ test) Final Mark = quizzes + assignments + exam (No Hurdle) 9Assignments Tentative We have three individual assignments Assignment 1: Data ingestion, cleaning manipulation and
Visualization: - 15 marks - Release Week3, due on the end of week 5. Assignment 2: Data Service (REST API): - 20 marks - Release on week 5, due on the end week 7. Assignment 3: Data Analytics Service: - 15 marks - Release on week 7, due on the end week 10. Bonus Mark We have 5 bonus marks on the assignments work overall mark. 10 Bonus Mark – 5 marks added to the assignments over all – Assignment over all= assignment1 + assignemnt2 + assignment3 +
Bonus – Assignment overall cannot be more than 50% – The weight of Bonus vary according to the contribution. How? ? Interesting ideas about doing the same activity with less complexity
(fewer lines of codes and more efficient) ? Improving the code (finding bugs, documentations, etc.) ? Adding new relevant activities or projects. ? Making a video for an activity and describing activities in detail ? Solving challenges announced during the lectures. 11 Consultation Labs ? A self-guided lab exercise is released every week.
? You can do them in your own time and come to the consultation
Labs if needed. ? Use the forum. Share what you have learned/found 12 Technologies Used this Term ? WebCMS for Announcements/Material
? Ed Stream for Discussions/Q&A ? Ms Teams for Live Lectures, Consultation Lab Sessions. ? Give for submission of Assignments 13 Tentative
Schedule Week Lectures Tutorials/Labs Assignments 1 Course Intro (No Lab, student should start by the
Setup Python, Flask, NumPy, Pandas) - 2 Data Access and ingestion Accessing NoSQL DB, API data sourced,
CSV files, text files. - 3 Data Cleansing and Manipulation Cleansing data with Python Pandas and
Open refine Assgn1 release 4 Data Visualization Using matplotlib library for charts and
plots 5 Building a Data service (part1) Build a simple Flask REST API Assgn1 due Release Ass2 6 --- --- --- 7 Building a Data service (part2) RESTful Client Assgn2 Due Release Assgn3 8 Data Analytics Applied Techniques and
Tools part1 Classification example 9 Data Analytics Applied Techniques and
Tools part2 Clustering example - 10 Final wrap-up - Assgn3 due 14 Supplementary Exam Policy Supp Exam is only available to students who: ? DID NOT attend the final exam ? Have a good excuse for not attending ? Have documentation for the excuse Submit special consideration within 72 hours (via myUNSW with supporting docs) Everybody gets exactly one chance to pass the final exam.
For CSE supplementary assessment
policy, follow the link in the course outline. 15 Student Conduct
請加QQ：99515681 郵箱：99515681@qq.com WX：codinghelp
