Paper Of The Week The Ml Test_score

social

Paper of the Week: “The ML Test Score: A Rubric for ML Production Readiness and Technical Debt Reduction”

ML-Based system testing and monitoring Average ML test scores for interviewed teams

How do you know if your ML system has enough tests? Has the right tests? Are you confident in your model and your pipeline? The authors of this paper, who are all Google employees, are proposing a rubic to tell if your ML tests are good enough. ML tests aren’t the same as normal software tests, since they need to cover the input data and because testing the behavior of the model is hard when you don’t know what the model will behave like! The authors propose four groups of tests: data tests (testing your input data), model development tests (ensuring your model is effective and has the impact you want), infrastructure tests (testing the pipeline and ensuring models can be rolled back), and monitoring tests (make sure the model isn’t stale, and that dependency changes throw alerts).

I’ll definitely be implementing some of these tests for my next project. I recommend checking out the paper, which you can read here Shout out to Austin #WiDS, who chose this paper for their paper discussion group!