Difference between revisions of "Test Tools The Next Generation"
PaulJulius (talk | contribs) (Formatting.) |
PaulJulius (talk | contribs) m (Formatting.) |
||
| (One intermediate revision by the same user not shown) | |||
| Line 39: | Line 39: | ||
** Issues related to the ephemeral nature of the infrastructure | ** Issues related to the ephemeral nature of the infrastructure | ||
* Costs | * Costs | ||
| − | + | ** of Execution | |
| − | + | ** of Maintenance | |
* Infrastructure | * Infrastructure | ||
| − | + | ** How to test the infrastructure | |
---------------- | ---------------- | ||
* What about something that enables getting end users involved? | * What about something that enables getting end users involved? | ||
| − | + | ** Session recording? | |
| − | + | ** Prototyping? | |
| − | + | ** Exploration toolkits | |
| − | + | ** Paper prototyping | |
| − | + | ** Goal: Get feedback from the customer as soon as possible to guide your development | |
* Mechanical Turk, how much can you control it? | * Mechanical Turk, how much can you control it? | ||
| − | + | ||
| − | PJ | + | PJ Proposing "AI Testing", using Machine Learning because the greatest cost in testing is maintaining a legacy test suite. |
| − | + | ||
| − | Proposing "AI Testing", using Machine Learning | ||
| − | because the greatest cost in testing is maintaining a legacy test suite. | ||
| − | |||
Mark Streibeck from Google on Testing | Mark Streibeck from Google on Testing | ||
| − | + | * If the test fails, and you have to change the test, then it is a BAD test | |
| − | + | * If the test fails and you change the code, the it is a GOOD test | |
Microsoft Reverse Code Coverage tool? | Microsoft Reverse Code Coverage tool? | ||
Latest revision as of 00:12, 12 September 2015
CITCON 2015 Europe in Helsinki
Session 1 - Auditorium
Testing Tools of the Future - Proposed by Gojko Adzic
Given how cheap resources (machines, people, etc), what can we do now that we could not do before.
Existing Examples: https://www.usertesting.com/
https://aws.amazon.com/device-farm/
Reminder of Jason Huggins's
- Why not just do production monitoring, like canary testing?
- It's a matter of risk
Third interesting trend: Webkits really grown up
Gojko uses Phantom for DOM testing all the time.
Wraith, from BBC, is a testing tool that runs the test in lots of browsers and then diff the visuals
Gatling, another tool that was very active
James Shore released Quixote, CSS Unit Testing, tool
Gauge was released by ThoughtWorks, as replacement for Twist
Classes of Problems:
- Fragmentation
- Issues of testing across multiple devices
- Resilience
- Issues related to the ephemeral nature of the infrastructure
- Costs
- of Execution
- of Maintenance
- Infrastructure
- How to test the infrastructure
- What about something that enables getting end users involved?
- Session recording?
- Prototyping?
- Exploration toolkits
- Paper prototyping
- Goal: Get feedback from the customer as soon as possible to guide your development
- Mechanical Turk, how much can you control it?
PJ Proposing "AI Testing", using Machine Learning because the greatest cost in testing is maintaining a legacy test suite.
Mark Streibeck from Google on Testing
- If the test fails, and you have to change the test, then it is a BAD test
- If the test fails and you change the code, the it is a GOOD test
Microsoft Reverse Code Coverage tool? To speed up feedback, by running
A/B Testing can be more difficult in Finance applications
One Company that Gojko knows of wrote a test, but as soon as it passes, they deleted the test
False positives, test designs
Is a "test suite" a _tall_ factory?
Pascal community used mathematical modeling
Google captures a list of system attributes/capabilities
Unit testing: if the unit test is hard to write, then that indicates a code smell
References
=====
Quixote, CSS Unit Testing Galen Framework Wraith, from BBC Device Farm from AWS - https://aws.amazon.com/device-farm/ UserTesting.com Gauge Concordian Applitools Visual Review Domreactor Hotjar Text Test - for testing log files
