AI DevWorld -- PRO Stage 4

Join on Hopin

Wednesday, October 28, 2020

- PDT
PRO SESSION (AI) : Automated Web Screen Collapse Detection by Graph Network Technology
Soichi Onozuka
Soichi Onozuka
IBM, IBM, Senior IT Specialist

An automated web application testing tool generates huge number of screenshots and rarely contains errors such as screen collapse.
Manual verification used be only way to find the error before I have successfully developed web screen error detection automatically, applying machine learning (ML) , specially “graph” technology. The uniqueness is, I applied ML to detect error in large and small area separately then they are merged to classify error screen image, this technique is called ensemble learning notably graph technology is best at capturing image structural feature in terms of its semantics.
I applied the following machine learning technologies, Random Forest and Semantic Graph Convolutional Networks to detect functional and non-functional error from screen images.
Applied frameworks are, Tensorflow-Keras, Scikit-learn and Networkx