Course Outline (W2019)

ELE888: Intelligent Systems

Instructor(s)Xiao-Ping Zhang [Coordinator]
Office: ENG451
Phone: (416) 979-5000 x 6686
Email: xzhang@ryerson.ca
Office Hours: TBA

Ling Guan
Office: ENG315
Phone: (416) 979-5000 x 6072
Email: lguan@ryerson.ca
Office Hours: TBA

Calendar DescriptionMachine learning and pattern classification are fundamental blocks in the design of an intelligent system. This course will introduce fundamentals of machine learning and pattern classification concepts, theories, and algorithms. Topics covered include: Bayesian decision theory, linear discriminant functions, multilayer neural networks, classifier evaluation, and an introduction to unsupervised clustering/grouping, self-organization and evolutionary computation.
PrerequisitesMTH 514




Compulsory Text(s):
  1. There are no required textbooks for this course. All of the material to be learned will be self-contained in the lecture notes that the instructor will provide as well as supplemental material to reinforce the concepts.

Reference Text(s):
  1. R. O. Duda, P. E. Hart and D. G. Stork, Pattern Classification, 2nd edition, John Wiley & Sons, 2002. ISBN: 0-471-05669-3.

Learning Objectives (Indicators)  

At the end of this course, the successful student will be able to:

  1. Generates solutions for complex engineering design problems (4b)
  2. Demonstrates iterative design process in complex engineering projects (4c)
  3. Constructs effective arguments and draws conclusions using evidence. Writes and revises documents using appropriate discipline specific conventions. Adapts format, content, organization, and tone for various audiences. Demonstrates accurate use of technical vocabulary. (7a)
  4. Constructs effective arguments and draws conclusions using evidence. Writes and revises documents using appropriate discipline specific conventions. Adapts format, content, organization, and tone for various audiences. Uses graphics to explain, interpret, and assess information. (7c)

NOTE:Numbers in parentheses refer to the graduate attributes required by the Canadian Engineering Accreditation Board (CEAB).

Course Organization

3.0 hours of lecture per week for 13 weeks
1.0 hours of lab/tutorial per week for 12 weeks

Teaching AssistantsTBA
Course Evaluation
Midterm Exam 30 %
Lab Reports 30 %
Final Exam 40 %
TOTAL:100 %

Note: In order for a student to pass a course with "Theory and Laboratory" components, in addition to earning a minimum overall course mark of 50%, the student must pass the Laboratory and Theory portions separately by achieving a minimum of 50% in the combined Laboratory components and 50% in the combined Theory components. Please refer to the "Course Evaluation" section for details on the Theory and Laboratory components.

ExaminationsMidterm exam, two hours, closed book (covers Weeks 1-6).
 Final exam, during exam period, three hours, closed-book (covers all course materials).
Other Evaluation InformationWill be informed if necessary
Other InformationNone

Course Content



Chapters /

Topic, description



Introduction and General Concepts of Machine Learning and AI systems



Introduction and General Concepts of Machine Learning and AI systems. Linear Algebra Review. Feature Scaling and Choice of Learning Rate - Gradient
 Descent. Gradient Descent for Large Datasets.



Review of Probability Concepts. Bayesian Decision Theory. Bayes
 Theorem and Decision Rule. Minimum Risk Action. Discriminant
 Functions. Gaussian Distributions.



Bayesian Decision Theory (continued).  Skewed Data Evaluation: Precision and Recall etc.



Linear Discriminant Functions



Linear Discriminant Functions/algorithms



Midterm Tutorials



midterm exam



Introduction of Multiplayer Neural Networks. BP algorithms. Deep learning.



Neural Networks Continued. Advice on using Neural
 Networks. Introduction to Support Vector Machines: Cost Function
 Kernels Optimizing Cost Function.
 Advice on Applying Machine Learning Algorithms. Bias and Variance.
 Learning Curves. Machine Learning System Design. Error Analysis.
 Classifier Evaluation.



Introduction to Unsupervised Learning. K-means Clustering.



Unsupervised Learning. Fuzzy C-means Clustering.
 Hierarchical Clustering.



Final Tutorials

Laboratory/Tutorials/Activity Schedule





Lab 0

Lab Assignment 0: Intro to matlab


Lab 1

Lab Assignment 1: Bayesian Decision Theory


Lab 2

Lab Assignment 2: Linear Discriminant Function


Lab 3

Lab Assignment 3:  Multilayer Neural Network


Lab 4

Lab Assignment 4:   Unsupervised Learnin

Policies & Important Information:

  1. Students are required to obtain and maintain a Ryerson e-mail account for timely communications between the instructor and the students;
  2. Any changes in the course outline, test dates, marking or evaluation will be discussed in class prior to being implemented;
  3. Assignments, projects, reports and other deadline-bound course assessment components handed in past the due date will receive a mark of ZERO, unless otherwise stated. Marking information will be made available at the time when such course assessment components are announced.
  4. Refer to our Departmental FAQ page for information on common questions and issues at the following link: https://www.ee.ryerson.ca/guides/Student.Academic.FAQ.html.

Missed Classes and/or Evaluations

When possible, students are required to inform their instructors of any situation which arises during the semester which may have an adverse effect upon their academic performance, and must request any consideration and accommodation according to the relevant policies as far in advance as possible. Failure to do so may jeopardize any academic appeals.

  1. Health certificates - If a student misses the deadline for submitting an assignment, or the date of an exam or other evaluation component for health reasons, they should notify their instructor as soon as possible, and submit a Ryerson Student Health Certificate AND an Academic Consideration Request form within 3 working days of the missed date. Both documents are available at https://www.ryerson.ca/senate/forms/medical.pdf.. If you are a full-time or part-time degree student, then you submit your forms to your own program department or school;
  2. Religious, Aboriginal and Spiritual observance - If a student needs accommodation because of religious, Aboriginal or spiritual observance, they must submit a Request for Accommodation of Student Religious, Aboriginal and Spiritual Observance AND an Academic Consideration Request form within the first 2 weeks of the class or, for a final examination, within 2 weeks of the posting of the examination schedule. If the requested absence occurs within the first 2 weeks of classes, or the dates are not known well in advance as they are linked to other conditions, these forms should be submitted with as much lead time as possible in advance of the absence. Both documents are available at www.ryerson.ca/senate/forms/relobservforminstr.pdf. If you are a full-time or part-time degree student, then you submit the forms to your own program department or school;
  3. Academic Accommodation Support - Before the first graded work is due, students registered with the Academic Accommodation Support office (AAS - www.ryerson.ca/studentlearningsupport/academic-accommodation-support) should provide their instructors with an Academic Accommodation letter that describes their academic accommodation plan.

Academic Integrity

Ryerson's Policy 60 (the Academic Integrity policy) applies to all students at the University. Forms of academic misconduct include plagiarism, cheating, supplying false information to the University, and other acts. The most common form of academic misconduct is plagiarism - a serious academic offence, with potentially severe penalties and other consequences. It is expected, therefore, that all examinations and work submitted for evaluation and course credit will be the product of each student's individual effort (or an authorized group of students). Submitting the same work for credit to more than one course, without instructor approval, can also be considered a form of plagiarism.

Suspicions of academic misconduct may be referred to the Academic Integrity Office (AIO). Students who are found to have committed academic misconduct will have a Disciplinary Notation (DN) placed on their academic record (not on their transcript) and will normally be assigned one or more of the following penalties:

  1. A grade reduction for the work, ranging up to an including a zero on the work (minimum penalty for graduate work is a zero on the work);
  2. A grade reduction in the course greater than a zero on the work. (Note that this penalty can only be applied to course components worth 10% or less, and any additional penalty cannot exceed 10% of the final course grade. Students must be given prior notice that such a penalty will be assigned (e.g. in the course outline or on the assignment handout);
  3. An F in the course;
  4. More serious penalties up to and including expulsion from the University.

The unauthorized use of intellectual property of others, including your professor, for distribution, sale, or profit is expressly prohibited, in accordance with Policy 60 (Sections 2.8 and 2.10). Intellectual property includes, but is not limited to:

  1. Slides
  2. Lecture notes
  3. Presentation materials used in and outside of class
  4. Lab manuals
  5. Course packs
  6. Exams

For more detailed information on these issues, please refer to the Academic Integrity policy(https://www.ryerson.ca/senate/policies/pol60.pdf) and to the Academic Integrity Office website (https://www.ryerson.ca/academicintegrity/).

Important Resources Available at Ryerson

  1. The Library (https://library.ryerson.ca/) provides research workshops and individual assistance. Inquire at the Reference Desk on the second floor of the library, or go to library.ryerson.ca/guides/workshops
  2. Student Learning Support(https://www.ryerson.ca/studentlearningsupport) offers group-based and individual help with writing, math, study skills and transition support, and other issues.