Appium
We used Appium for 2 years in general and for 8 months with React Native, and found that we invested an unreasonable portion of our time writing tests and petting the system than actually writing features.
We found that End to End testing is really hard:
We found that End to End testing is really hard:
Tests are flaky, we got different results on different machines, frequent failures in CI, which the only solution for was addition of sleeps, which slowed the tests down.
Tests were already slow since Apple UIAutomation tool is limited to performing one action per second, and there’s a hack which removes this cap Instruments without delays (which is already unmaintained), so after each release of a new Xcode we would have to wait for patch before upgrading.
Detox Gray box, not black box
Detox does Gray box, not Black box,
 |
| A Black Box Testing |
to allows the test framework to monitor the app from the inside and actually synchronize with it.
Gray box essentially uses a piece of code that is planted in the app, it can help us see what’s going on inside.
Unlike Black box, Gray box runs in the same process, has access to memory, and can monitor the execution process. Being able to read internal memory gives it the ability to detect what’s happening inside the process: if there are network requests in flight, when the main thread is idle, other threads are idle, Animations have ended, the react native bridge is idle. It can execute on main thread, to make sure that when it performs actions nothing in the UI hierarchy changes in the meantime.