1. 簡(jiǎn)介

在探索App自動(dòng)化測(cè)試工具過(guò)程中偏序，主要接觸了 Macaca 和 Appium, 已及稍稍看了點(diǎn) Calabash。

其中咧叭，Calabash 與 Appium都支持使用 Cucumber 編寫(xiě)測(cè)試用例葱跋。由于精力有限，Calabash沒(méi)有更進(jìn)一步研究崔步，有興趣的朋友可以看看稳吮，交流一下心得。

Macaca在 敏捷實(shí)踐 - 我們是如何自動(dòng)化App驗(yàn)收標(biāo)準(zhǔn)（一） 一文中有提到過(guò)井濒，是我的前端小伙伴L(zhǎng)orne向我推薦的灶似，看了文檔，也裝上試了試瑞你，不得不說(shuō)Macaca非常優(yōu)秀酪惭，上手非常容易，同時(shí)支持 NodeJS 與 Python 與 Java 三大主流語(yǔ)言者甲。就是文檔稍有些少春感。推薦大家試試。

Appium是個(gè)非常優(yōu)秀的App自動(dòng)化測(cè)試工具虏缸，支持的語(yǔ)言比Macaca多一些鲫懒。讓我選擇Appium而不是Macaca的關(guān)鍵因數(shù)并非所支持語(yǔ)言的多少，而是Appium支持用Ruby+Cucumber來(lái)寫(xiě)測(cè)試用例刽辙，簡(jiǎn)直就像為我們量身定造刀疙。沒(méi)錯(cuò)，我最喜歡用的語(yǔ)言首選Ruby扫倡，其次NodeJS/Python谦秧，再之Java。

我們的測(cè)試用例起步于 appium sample-code 中的ruby部分撵溃，強(qiáng)烈推薦大家去翻翻代碼疚鲤。

2. Appium

Appium的詳細(xì)介紹信息可以參考 在線文檔，在這里我并不打算太多重復(fù)缘挑，而且網(wǎng)上也有不少教程了集歇，只想概要的說(shuō)明一下它是如何工作的。

2.1 Appium的體系結(jié)構(gòu)

Appium是用NodeJS寫(xiě)的一個(gè)HTTP服務(wù)语淘，因此使用它需要先安裝NodeJS诲宇。它又是通過(guò)WebDriverAgent組件在移動(dòng)設(shè)備中控制被測(cè)試應(yīng)用執(zhí)行測(cè)試指令际歼。

2.2 工作流程圖

注意，這個(gè)流程圖并非官方圖姑蓝，只是我隨手畫(huà)的幫助大家理解的示意圖鹅心。

appium_flow.png

測(cè)試腳本與Appium與被測(cè)試的APP是分別各自獨(dú)立的進(jìn)程，甚至分署在不同的主機(jī)與真實(shí)設(shè)備中纺荧。

測(cè)試的執(zhí)行過(guò)程為

1). 測(cè)試腳本調(diào)用appium_lib提供的方法, 如 find, click
2). appium_lib調(diào)用轉(zhuǎn)給Selenium WebDriver的Remote Adapter
3). Selenium WebDriver將請(qǐng)求封裝為JSON旭愧，以HTTP Rest協(xié)議向Appium發(fā)送請(qǐng)求
4). Appium收到HTTP請(qǐng)求后，將請(qǐng)求交給相應(yīng)的AppiumDriver
5). AppiumDriver通過(guò)HTTP向運(yùn)行在終端設(shè)備上的WebDriverAgent進(jìn)程(后臺(tái)app)發(fā)送請(qǐng)求
6). 設(shè)備上的WebDriverAgent解析請(qǐng)求后宙暇，通過(guò)XCUITest測(cè)試套件與目標(biāo)測(cè)試App交互输枯，并獲取結(jié)果
7). WebDriverAgent將結(jié)果由HTTPResponse返回給Appium
8). Appium再將結(jié)果進(jìn)行包裝處理后，由HTTPResponse返回給3)的Selenium WebDriver
9). Selenium WebDriver再將結(jié)果轉(zhuǎn)換成Ruby對(duì)象返回給測(cè)試腳本占贫。  

2.3 Appium常用指令

官方已經(jīng)有了非常完整的文檔桃熄，我也就不再重復(fù)或做些翻譯工作了。
Ruby語(yǔ)言請(qǐng)移步 https://github.com/appium/ruby_lib/blob/master/docs/docs.md

--

Example use of Appium's mobile gesture.

@driver.find_element()

console.rb uses some code from simple_test.rb and is released under the same license as Appium. The Accessibility Inspector is helpful for discovering button names and textfield values.

Long click on an ImageView in Android.

last_image = find_elements(:tag_name, :ImageView).last
long_press(element: last_image)

Rotate examples.

driver.rotate :landscape
driver.rotate :portrait

• status["value"]["build"]["revision"] Discover the Appium rev running on the server.
• driver.keyboard.send_keys "msg" Sends keys to currently active element

generic

• source Prints a JSON view of the current page.
• page Prints the content descriptions and text values on the current page.
• page_class Prints the classes found on the current page.
• (Element) find(value) Returns the first element that contains value.
• (Element) finds(value) Returns all elements containing value (iOS only for now).
• (Element) name(name) Returns the first element containing name. Android name is the content description.
  iOS uses accessibility label with a fallback to text.
• (Array<Element>) names(name) Returns all elements containing name.
• (Element) text(text) Returns the first element containing text.
• (Array<Element>) texts(text) Returns all elements containing text.
• current_app Returns information about the current app. Android only.

--

alert

1. (void) alert_accept Accept the alert.
2. (String) alert_accept_text Get the text of the alert's accept button.
3. (void) alert_click(value) iOS only Tap the alert button identified by value.
4. (void) alert_dismiss Dismiss the alert.
5. (String) alert_dismiss_text Get the text of the alert's dismiss button.
6. (String) alert_text Get the alert message text.

button

1. (Button) button(index) Find a button by index.
2. (Button) button(text) Get the first button that includes text.
3. (Array<String>, Array<Buttons>) buttons(text = nil) Get an array of button texts or button elements if text is provided.
4. (Array<Button>) buttons(text) Get all buttons that include text.
5. (Button) first_button Get the first button element.
6. (Button) last_button Get the last button element.

textfield

1. (Textfield) textfield(index) Find a textfield by index.
2. (Array<Textfield>) textfields Get an array of textfield elements.
3. (Textfield) first_textfield Get the first textfield element.
4. (Textfield) last_textfield Get the last textfield element.
5. (Textfield) textfield_exact(text) Get the first textfield that matches text.
6. (Textfield) textfield(text) Get the first textfield that includes text.

text

The Static Text methods have been prefixed with s_ to avoid conflicting with the generic text methods.

1. (Text) text(index) Find a text by index.
2. (Array<Text>) texts Get an array of text elements.
3. (Text) first_text Get the first text element.
4. (Text) last_text Get the last text element.
5. (Text) text_exact(text) Get the first element that matches text.
6. (Text) text(text) Get the first textfield that includes text.

window

1. (Object) window_size Get the window's size.

--

e.name # button, text
e.value # secure, textfield
e.type 'some text' # type text into textfield
e.clear # clear textfield
e.tag_name # calls .type (patch.rb)
e.text
e.size
e.location
e.rel_location
e.click
e.send_keys 'keys to send'
e.set_value 'value to set' # ruby_console specific
e.displayed? # true or false depending if the element is visible
e.selected? # is the tab selected?
e.attribute('checked') # is the checkbox checked?


# alert example without helper methods
alert = $driver.switch_to.alert
alert.text
alert.accept
alert.dismiss

# Secure textfield example.
#
# Find using default value
s = textfield 'Password'
# Enter password
s.send_keys 'hello'
# Check value
s.value == ios_password('hello'.length)

--

Driver

start_driver will restart the driver.

x will quit the driver and exit Pry.

execute_script calls $driver.execute_script

find_element calls $driver.find_element

find_elements calls $driver.find_elements

no_wait will set implicit wait to 0. $driver.manage.timeouts.implicit_wait = 0

set_wait will set implicit wait to default seconds. $driver.manage.timeouts.implicit_wait = default

set_wait(timeout_seconds) will set implicit wait to desired timeout. $driver.manage.timeouts.implicit_wait = timeout

.click to tap an element.
.send_keys to type on an element.

Raw UIAutomation

execute_script "au.lookup('button')[0].tap()" is the same as
execute_script 'UIATarget.localTarget().frontMostApp().buttons()[0].tap()'

See app.js for more au methods.
Note that raw UIAutomation commands are not officially supported.

Advanced au.

In this example we lookup two tags, combine the results, wrap with $, and then return the elements.

s = %(
var t = au.lookup('textfield');
var s = au.lookup('secure');
var r = $(t.concat(s));
au._returnElems(r);
)

execute_script s

XPath(UIAutomation)

See #194 for details.

find_element  :xpath, 'button'
find_elements :xpath, 'button'

find_element  :xpath, 'button[@name="Sign In"]'
find_elements :xpath, 'button[@name="Sign In"]'

find_element  :xpath, 'button[contains(@name, "Sign In")]'
find_elements :xpath, 'button[contains(@name, "Sign")]'

find_element :xpath, 'textfield[@value="Email"]'
find_element :xpath, 'textfield[contains(@value, "Email")]'

find_element  :xpath, 'text[contains(@name, "Reset")]'
find_elements :xpath, 'text[contains(@name, "agree")]'

3. Appium支持與無(wú)法支持的測(cè)試

在 2.2 中型奥，可以看到測(cè)試代碼并不能直接訪問(wèn)被測(cè)試的App已及它的控件瞳收。因此能測(cè)試的也就是XCUITest (iOS)套件所支持的方法與屬性。

支持的測(cè)試:
查找元素桩引，獲取name value type text size location enabled? displayed? selected?等屬性，或者發(fā)送點(diǎn)擊事件收夸，為文本框輸入文字坑匠。

這些基本上都屬于功能性測(cè)試，可以用來(lái)操作輸入框卧惜，點(diǎn)擊按鈕厘灼，查找某段文字是否存在。

如

1) 打開(kāi)登錄頁(yè)面
2) 找到 “用戶名” 輸入框咽瓷，輸入 test@test.com
3) 找到 “密碼” 輸入框设凹， 輸入 123456
4) 找到 “登錄” 按鈕，向它發(fā)生點(diǎn)擊事件
5) 登錄成功茅姜，檢查用戶是否處于主頁(yè)面

傳送門(mén)
https://github.com/appium/sample-code/blob/master/sample-code/examples/ruby/simple_test.rb

# GETTING STARTED
# -----------------
# This documentation is intended to show you how to get started with a
# simple Appium & appium_lib test.  This example is written without a specific
# testing framework in mind;  You can use appium_lib on any framework you like.
#
# INSTALLING RVM
# --------------
# If you don't have rvm installed, run the following terminal command
#
# \curl -L https://get.rvm.io | bash -s stable --ruby
#
# INSTALLING GEMS
# ---------------
# Then, change to the example directory:
#   cd appium-location/sample-code/examples/ruby
#
# and install the required gems with bundler by doing:
#   bundle install
#
# RUNNING THE TESTS
# -----------------
# To run the tests, make sure appium is running in another terminal
# window, then from the same window you used for the above commands, type
#
# bundle exec ruby simple_test.rb
#
# It will take a while, but once it's done you should get nothing but a line
# telling you "Tests Succeeded";  You'll see the iOS Simulator cranking away
# doing actions while we're running.
require 'rubygems'
require 'appium_lib'

APP_PATH = '../../apps/TestApp/build/release-iphonesimulator/TestApp.app'

desired_caps = {
  caps:       {
    platformName:  'iOS',
    versionNumber: '8.1',
    deviceName:    'iPhone 6',
    app:           APP_PATH,
  },
  appium_lib: {
    sauce_username:   nil, # don't run on Sauce
    sauce_access_key: nil
  }
}

# Start the driver
Appium::Driver.new(desired_caps).start_driver

module Calculator
  module IOS
    # Add all the Appium library methods to Test to make
    # calling them look nicer.
    Appium.promote_singleton_appium_methods Calculator

    # Add two numbers
    values       = [rand(10), rand(10)]
    expected_sum = values.reduce(&:+)

    # Find every textfield.
    elements     = textfields

    elements.each_with_index do |element, index|
      element.type values[index]
    end

    # Click the first button
    button(1).click

    # Get the first static text field, then get its text
    actual_sum = first_text.text
    raise unless actual_sum == (expected_sum.to_s)

    # Alerts are visible
    button('show alert').click
    find_element :class_name, 'UIAAlert' # Elements can be found by :class_name

    # wait for alert to show
    wait { text 'this alert is so cool' }

    # Or by find
    find('Cancel').click

    # Waits until alert doesn't exist
    wait_true { !exists { tag('UIAAlert') } }

    # Alerts can be switched into
    wait { button('show alert').click } # Get a button by its text
    alert         = driver.switch_to.alert # Get the text of the current alert, using
    # the Selenium::WebDriver directly
    alerting_text = alert.text
    raise Exception unless alerting_text.include? 'Cool title'
    alert_accept # Accept the current alert

    # Window Size is easy to get
    sizes = window_size
    raise Exception unless sizes.height == 667
    raise Exception unless sizes.width == 375

    # Quit when you're done!
    driver_quit
    puts 'Tests Succeeded!'
  end
end

提醒闪朱，由于sample-code中的TestApp比較老了，上面的測(cè)試代碼直接跑钻洒，是會(huì)失敗的奋姿。這個(gè)是很隱晦的問(wèn)題導(dǎo)致，新版模擬器(eg. iPhone 6 (iOS 10.0x)), 在運(yùn)行這個(gè)app是素标，會(huì)自動(dòng)彈出一個(gè)告警信息称诗，說(shuō)“TestApp使用老版本編譯的，會(huì)影響系統(tǒng)系能头遭，建議用新xcode重新編譯”寓免。 就是這個(gè)alert框癣诱，導(dǎo)致測(cè)試腳本找不到相應(yīng)的控件而失敗。

這個(gè)事情一開(kāi)始也卡了我很久袜香，為什么測(cè)試用例會(huì)通不過(guò)撕予？后來(lái)是在用Appium Ruby Console的時(shí)候找到的。

解決方式有兩個(gè)：
一困鸥、重新編譯TestApp;

二嗅蔬、測(cè)試代碼在“ # Add two numbers” 前加入 sleep(10) 暫停10秒，讓你有機(jī)會(huì)點(diǎn)擊 alert 上的 OK 按鈕疾就，關(guān)掉對(duì)話框澜术。

當(dāng)時(shí)不想多事，選擇了二猬腰。
現(xiàn)在想想鸟废，應(yīng)該還有三： 就是把sleep 換成 alert_accept 來(lái)關(guān)掉對(duì)話框。 有興趣的可以自己試試姑荷。

無(wú)法支持的測(cè)試:
首先盒延，測(cè)試腳本無(wú)法直接訪問(wèn)被測(cè)對(duì)象，其次鼠冕，來(lái)之于上面XCUITest的限制添寺，關(guān)于控件有無(wú)邊框，控件狀態(tài)懈费，動(dòng)畫(huà)效果计露，字體變化，顏色憎乙，對(duì)齊等視覺(jué)效果票罐，我們是無(wú)法測(cè)試的。

關(guān)于這一點(diǎn)泞边，stackoverflow 上也有老外在吐槽：
http://stackoverflow.com/questions/31250941/xcuielement-obtain-image-value

http://www.danielhall.io/exploring-the-new-ui-testing-features-of-xcode-7

如该押，

a) 文章的標(biāo)題需要根據(jù)不同的狀態(tài)，用紅色阵谚，黑色蚕礼，灰色等不同的顏色展示。

b) 當(dāng)點(diǎn)擊“上傳”按鈕后梢什，頭像Image應(yīng)當(dāng)加載新的頭像地址闻牡。

這種測(cè)試通常就屬于無(wú)法用自動(dòng)化來(lái)檢驗(yàn)的。

尤其是我們的App是用React Native開(kāi)發(fā)的绳矩，給我們的測(cè)試用例帶來(lái)了更多的限制與挑戰(zhàn)罩润。

4. 解決方案

自動(dòng)化測(cè)試更多的是在功能性上，流程性上的測(cè)試起作用翼馆，基本能覆蓋了80%～90%的功能性測(cè)試范圍割以。

對(duì)于自動(dòng)化不好做的AC金度，我們的處理方式很簡(jiǎn)單，把它們放到人工測(cè)試的范圍去严沥，當(dāng)環(huán)境猜极、需求、技術(shù)發(fā)生變化了消玄，能自動(dòng)化測(cè)試了跟伏，再自動(dòng)化，不要太去糾結(jié)翩瓜。

最后受扳，自動(dòng)化測(cè)試并不能代替一切，也無(wú)法覆蓋所有的地方兔跌。
它的最大價(jià)值在于使得App的開(kāi)發(fā)也能夠持續(xù)集成勘高。
極大的降低了全回歸的成本與工作量。