Skip to end of metadata
Go to start of metadata

You are viewing an old version of this page. View the current version.

Compare with Current View Page History

« Previous Version 4 Next »

11 August 2020 - 12pm Eastern

Available test environments: updates from Stella:

  • BrowserStack & SauceLabs licenses in place

  • Perfecto is also back online - with a new URL

“Hello world” Mobile test has made good progress (Liz):

  • Basic “hello world” integration is up with BrowserStack

  • For SauceLabs, open request still for webdriver.io - no useful support yet.

Tech stack (Liz)

  • Assertion library is Jasmine, similar to Jest.

  • Outstanding tooling choices: webdriver vs. nightwatch vs. testcafe vs. cypress other options.

  • Not a priority to have aligned stacks between mobile app & Safe Places, as they willl have a completely separate set of test cases.

  • We are using webdriver with Appium, as we don’t believe other options work with Appium. Choice still wide open for Web / Selenium.

Next steps on Mobile Automation:

  • Test succeeds but still returns an error code: this is step #1

  • step #2: get an iOS “hello world” working.

  • Not yet automated into pipeline - step #3.

  • Then we could begin implementing basic test cases.

  • Liz also suggests we look at WDIO image comparison service & native app compare - screen comparison tools. Easy to set up & powerful. Jeri will have a look at this over the next week.

This week:

  • Get Jeri up & running to match Liz’s level

  • Steps #1, #2 above: Android & iOS working

  • Start looking at pipeline integration & writing more tests.

Other items (Deven):

  • Z Attack Proxy (ZAP) - can analyze Selenium tests & spot any insecure practices on the fly.

  • ZAP HUD, feedback to developers in real-time.

  • Deven has some training materials Stella will take a look at this.

4 August 2020 - 12pm Eastern

Liz trying to get Appium running against Sauce Labs, but not yet working.

  • Stella is working on set-up with Sauce Labs.

  • Suspects that some of the automation pieces are not yet set up.

  • (also Perfecto is not working at the moment, but we hope it will come back).

  • Liz should route issues through Stella to get attention from SL - send errors etc.

With Web Driver, Perfecto has a Java driver, but no JS driver. SauceLabs has both.

  • We prefer to develop in JS for consistency with Dev team.

BrowserStack is another option Liz could look at (also Jonathon’s private device cloud if he can provide some docs on this).

We don’t have iOS build automatically generated at the moment. We’re not sure why not

  • Liz to focus on Android for now.

  • Then will look at fixing up iOS builds later.

“Hello world” test just checks whether the app is running.

  • Focus on running locally first - yarn install to set up environment

  • Sauce Connect tunnel is the only fiddly bit, currently a few manual steps. Liz looking at using a Docker container to streamline this. Some input from Sauce Labs would be useful here.

  • Then wire this into CI once it is running manually.

Goals for next week:

  • Hello world test, launch app, close app. With one of these cloud options (Perfecto, SL, BrowserStack or Jonathon’s setup).

  • Should work on both Windows & Mac.

  • Jeri to be able to run the tests as well (Mac)

  • Diarmid should be able to run it on Windows as well.

28 July 2020 - 12pm Eastern

  • Consensus that we won’t move forwards with Eggplant. Significant investment needed to fix up the model to match the latest app, and we don’t see much value in return.

  • 21labs also needs significant work to update models to work with the latest app. We think there is more value here, but we’re unsure whether we wan to move forwards with 21labs, or whether Native Appium would be better,

  • 21labs vs. Native Appium

    • 21labs is more accessible for folks who can’t code. But for people comfortable with coding, it’s slower, fiddlier, and harder to learn than coding with Appium (we suspect).

    • 21labs automatic maintenance of scripts is a neat feature, but if we design code for Appium scripts well, with good re-use of code, the problem it solves won’t manifest in the first place.

    • Currently, we have 3 people (Liz, Jeri, Kallie) all comfortable with coding, and happy to learn Appium - and have found 21labs frustrating, fiddly & slow.

    • We agreed to have a spike where we try to push forwards with test scripts in Appium directly (running against Perfecto). After a week, we’ll re-assess. If things go well we’ll likely stick with Appium. If they go badly, we may pivot back to 21labs.

    • Native Appium may run a bit faster than 21labs, which may help us with CI costs - see later point.

  • Plan for the week

    • Goal for next Tues (4 August) was to have a basic Appium script that we could run that gets us through onboarding.

    • (once we can do that, the rest of the app is just more button pushes, so should be pretty easy).

    • (we didn’t explicitly specify iOS or Android. Either would do. Both would be awesome!)

  • Other points: e2e testing

    • For e2e testing, we’ll need to figure out how to extract codes for location data sharing from Safe Places.

    • Various options:

      • Direct access to Safe Places Back End API - simplest if we can.

      • Using Appium + browser on the Mobile App to access Safe Places Front End.

      • Invoke some Selenium scripts (which will probably be developed anyway by the Selenium team)

    • Solving this problem is a priority for next week.

  • Other points: CI/CD

    • Once we have some working scripts, we’ll want to integrate into the CI/CD pipeline.

    • Let’s do this as soon as we have something of some minimal value.

    • TBD how often this runs. Bare minimum would be once a week, manually.

    • Ideally it would run much more frequently, potentially every build.

    • If the tests take a long time, this could increase out GitHub CI costs, as the VM that does the builds sits waiting for the results of the automated tests to return. This is a future challenge we will need to address if we want to get these tests running on a very frequent basis.

  • No labels