From System Logs to System Optimization with the Power of Data Science and Machine Learning
- TBA
Held on June 10, 2019 at Argonne National Laboratory, this seminar covered the application of machine learning to data extraction and knowledge acquisition.
Combining, correlating and then analyzing multi-dimensional logs of one computer system using data mining and machine learning techniques provides a systematical insight into the system without adding extra instrumenting burden. It also has the potential to yield new insights that can help the co-design of hardware and system tools, optimize existing system tools and improve the performance and experience for end users. Data transfer nodes are computer systems purposely built for wide area file transfer. Here, we put this log co-analyze idea into practice for wide area file transfer service. Our practical work covers the full-stack of providing insights to explain performance, building models to predict performance and proposing methods to optimize the system performance.
In this talk, Zhengchun Liu, Research Scientist at Argonne & Joint Appointee, The University of Chicago, first shared success stories on how we extract information from massive logs and transfer information into knowledge for optimization using machine learning techniques. He then shared his work with a similar idea but for a light source facility in order to advance imaging experiments.