Machine Learning (ML) has shown a substantial impact on computational sciences in recent years. The adaptation of ML techniques to deal with various systems in physical sciences has gained ground in addition to the existing numerical methods. In this work, we introduce the readers to machine learning with special reference to Artificial Neural Networks (ANNs) that can solve ordinary differential equations (ODEs) and partial differential equations (PDEs) including those which are subject to specific symmetries. This paper will be helpful for graduate and undergraduate students as introductory material to early career researchers interested in applying ML techniques to solve problems in computational sciences. In particular, we choose elementary differential equations that describe systems from various fields of science to illustrate the proficiency of ANNs to capture the regularities that underlie such systems in the hope of adding ML techniques to the physicists’ toolbelt.