Selenium入门04

接上一节,说一下Grid的用法

1、启动hub

set JAVA_HOME=C:\NeoLanguages\Java\JDK\jdk_x86_1.8.0_77
set PATH=%JAVA_HOME%\bin;%PATH%;C:\NeoTest\TestSelenium\trunk\bin\x86\;
set webdriver.gecko.driver=C:\NeoTest\TestSelenium\trunk\bin\x86\geckodriver.exe

java -jar ../lib/selenium-server-standalone-3.3.1.jar -role hub -hubConfig hubConfig.json -debug true

pause

hubConfig.json

{
  "port": 4444,
  "newSessionWaitTimeout": -1,
  "servlets" : [],
  "withoutServlets": [],
  "custom": {},
  "capabilityMatcher": "org.openqa.grid.internal.utils.DefaultCapabilityMatcher",
  "throwOnCapabilityNotPresent": true,
  "cleanUpCycle": 5000,
  "role": "hub",
  "debug": false,
  "browserTimeout": 0,
  "timeout": 1800
}

2、启动node

set JAVA_HOME=C:\NeoLanguages\Java\JDK\jdk_x86_1.8.0_77
set PATH=%JAVA_HOME%\bin;%PATH%;C:\NeoTest\TestSelenium\trunk\bin\x86\;
set webdriver.gecko.driver=C:\NeoTest\TestSelenium\trunk\bin\x86\geckodriver.exe

java -jar ../lib/selenium-server-standalone-3.3.1.jar -role node -nodeConfig node01Config.json -debug true

pause

node01Config.json

{
 "capabilities":
  [
    {
      "browserName": "firefox",
      "maxInstances": 5,
      "seleniumProtocol": "WebDriver"
    }
  ],
  "proxy": "org.openqa.grid.selenium.proxy.DefaultRemoteProxy",
  "maxSession": 5,
  "port": 5555,
  "register": true,
  "registerCycle": 5000,
  "hub": "http://192.168.130.178:4444",
  "nodeStatusCheckTimeout": 5000,
  "nodePolling": 5000,
  "role": "node",
  "unregisterIfStillDownAfter": 60000,
  "downPollingLimit": 2,
  "debug": false,
  "servlets" : [],
  "withoutServlets": [],
  "custom": {}
}

3、可以在hub中看到注册情况
http://localhost:4444/grid/console
这个地方一定要注意,每个node上的浏览器及操作系统信息,向hub发送指令时,必须符合该信息

4、运行脚本
TestHelloWorldHub.py

# !C:\Languages\Python\Python27\python.exe
# -*- coding: utf-8 -*-
'''
Created on 2016-10-22
@author: Hansen
HelloWorld sample for NeoSelenium
'''

from NeoSelenium import initEngine
from NeoSelenium import deInitEngine
from NeoSelenium import initEngineHub

#一个简单的查询测试
def neohope_search_test():
    try:
        #myEngine = initEngine('ie32')
        #myEngine = initEngine('ie64')
        #myEngine = initEngine('chrome')
        #myEngine = initEngine('ff32')
        #myEngine = initEngine('ff64')
        #myEngine = initEngineRemote()
	myEngine = initEngineHub()

        base_url = "https://www.neohope.org"
        myEngine.get(base_url + "/")
        myEngine.find_element_by_name("s").clear()
        myEngine.find_element_by_name("s").send_keys("Metabase")
        myEngine.find_element_by_css_selector("button.search-submit").click()
        myEngine.implicitly_wait(1000)
        #print(myEngine.find_elements_by_xpath("//div[@id='content']/article"))
        queryResultLenght = len(myEngine.find_elements_by_xpath("//div[@id='content']/article"))
        #print(queryResultLenght)
        #应该是1但现在是10
        assert queryResultLenght==1
    finally:
        #deInitEngine(myEngine)
        print("test end")

#start here
neohope_search_test()

NeoSelenium.py

#!C:\Languages\Python\Python27\python.exe
# -*- coding: utf-8 -*-
'''
Created on 2017-04-07
@author: Hansen
NeoSelenium
'''

import os
from selenium import webdriver
from selenium.webdriver.common.desired_capabilities import DesiredCapabilities

NPath = None
x86Path = "C:\NeoTest\TestSelenium\\trunk\\bin\\x86;"
x64Path = "C:\NeoTest\TestSelenium\\trunk\\bin\\x64;"

'''
selenium remote engine init
'''
def initEngineRemote():
    engine = webdriver.Remote(command_executor="http://192.168.130.178:4444/wd/hub", desired_capabilities=DesiredCapabilities.FIREFOX)
    return engine

'''
selenium remote engine init
'''
def initEngineHub():
    engine = webdriver.Remote(command_executor="http://192.168.130.178:5555/wd/hub", desired_capabilities={
        "browserName": "firefox",
        "platform": "VISTA"
    })
    return engine   

'''
selenium engine deInit
'''
def deInitEngine(engine):
    engine.close()
    engine.quit()


Selenium入门03

接上一节,说一下如何进行远程测试

1、下载selenium-server-standalone的jar包
http://www.seleniumhq.org/download/

2、运行selenium-server-standalone

set JAVA_HOME=C:\NeoLanguages\Java\JDK\jdk_x86_1.8.0_77
set PATH=%JAVA_HOME%\bin;%PATH%;C:\NeoTest\TestSelenium\trunk\bin\x86\;
set webdriver.gecko.driver=C:\NeoTest\TestSelenium\trunk\bin\x86\geckodriver.exe

java -jar ../lib/selenium-server-standalone-3.3.1.jar -role standalone

pause

3、远程测试脚本
TestHelloWorldRemote.py

# !C:\Languages\Python\Python27\python.exe
# -*- coding: utf-8 -*-
'''
Created on 2016-10-22
@author: Hansen
HelloWorld sample for NeoSelenium
'''

from NeoSelenium import deInitEngine
from NeoSelenium import initEngineRemote

#一个简单的查询测试
def neohope_search_test():
    try:
        #myEngine = initEngine('ie32')
        #myEngine = initEngine('ie64')
        #myEngine = initEngine('chrome')
        #myEngine = initEngine('ff32')
        #myEngine = initEngine('ff64')
	myEngine = initEngineRemote()

        base_url = "https://www.neohope.org"
        myEngine.get(base_url + "/")
        myEngine.find_element_by_name("s").clear()
        myEngine.find_element_by_name("s").send_keys("Metabase")
        myEngine.find_element_by_css_selector("button.search-submit").click()
	myEngine.implicitly_wait(1000)
	#print(myEngine.find_elements_by_xpath("//div[@id='content']/article"))
	queryResultLenght = len(myEngine.find_elements_by_xpath("//div[@id='content']/article"))
	#print(queryResultLenght)
	#应该是1但现在是10
        assert queryResultLenght==1
    finally:
        #deInitEngine(myEngine)
	print("test end")

#start here
neohope_search_test()

NeoSelenium.py

#!C:\Languages\Python\Python27\python.exe
# -*- coding: utf-8 -*-
'''
Created on 2017-04-07
@author: Hansen
NeoSelenium
'''

import os
from selenium import webdriver
from selenium.webdriver.common.desired_capabilities import DesiredCapabilities

NPath = None
x86Path = "C:\NeoTest\TestSelenium\\trunk\\bin\\x86;"
x64Path = "C:\NeoTest\TestSelenium\\trunk\\bin\\x64;"

'''
selenium remote engine init
'''
def initEngineRemote():
    engine = webdriver.Remote(command_executor="http://192.168.130.178:4444/wd/hub", desired_capabilities=DesiredCapabilities.FIREFOX)
    return engine

'''
selenium engine deInit
'''
def deInitEngine(engine):
    engine.close()
    engine.quit()

Selenium入门02

下面就说一下如何用Python进行自动化测试咯

1、安装语言包

pip install selenium

2、在selenium网站下载对应浏览器的driver
http://www.seleniumhq.org/download/

3、改一下我写的这个文件,将driver路径改成正确的路径
NeoSelenium.py

#!C:\Languages\Python\Python27\python.exe
# -*- coding: utf-8 -*-
'''
Created on 2017-04-07
@author: Hansen
NeoSelenium
'''

import os
from selenium import webdriver
from selenium.webdriver.common.desired_capabilities import DesiredCapabilities

NPath = None
x86Path = "C:\NeoTest\TestSelenium\\trunk\\bin\\x86;"
x64Path = "C:\NeoTest\TestSelenium\\trunk\\bin\\x64;"

def initIE32():
    global x86Path
    global NPath
    os.environ["PATH"] = x86Path+NPath
    #print(os.environ["PATH"])
    engine = webdriver.Ie()
    return engine

def initIE64():
    global x64Path
    global NPath
    os.environ["PATH"] = x64Path+NPath
    #print(os.environ["PATH"])
    engine = webdriver.Ie()
    return engine

def initChrome32():
    global x86Path
    global NPath
    os.environ["PATH"] = x86Path+NPath
    engine = webdriver.Chrome()
    return engine

def initFirfox32():
    global x86Path
    global NPath
    os.environ["PATH"] = x86Path+NPath
    #print(os.environ["PATH"])
    engine = webdriver.Firefox()
    return engine

def initFirfox64():
    global x64Path
    global NPath
    os.environ["PATH"] = x64Path+NPath
    #print(os.environ["PATH"])
    engine = webdriver.Firefox()
    return engine

'''
selenium engine init
'''
def initEngine(engineType):
    global NPath
    if(NPath is None):
        NPath = os.environ["PATH"]
    if(cmp('IE32',engineType.upper())==0):
        return initIE32()
    elif(cmp('IE64',engineType.upper())==0):
        return initIE64()
    elif(cmp('CHROME',engineType.upper())==0 or cmp('CHROME32',engineType.upper())==0):
        return initChrome32()
    elif(cmp('FIREFOX',engineType.upper()) or cmp('FIREFOX32',engineType.upper()) or cmp('FF',engineType.upper())==0 or cmp('FF32',engineType.upper())==0):
        return initFirfox32()
    elif(cmp('FIREFOX64',engineType.upper()) or cmp('FF64',engineType.upper())==0):
        return initFirfox64()
    else:
        print("engin type: "+engineType.upper()+" not supported!")
        return None

'''
selenium remote engine init
'''
def initEngineRemote():
    engine = webdriver.Remote(command_executor="http://localhost:4444/wd/hub", desired_capabilities=DesiredCapabilities.FIREFOX)
    return engine
    

'''
selenium engine deInit
'''
def deInitEngine(engine):
    engine.close()
    engine.quit()

4、第一个自动化测试脚本
TestHelloWorld.py

# !C:\Languages\Python\Python27\python.exe
# -*- coding: utf-8 -*-
'''
Created on 2016-10-22
@author: Hansen
HelloWorld sample for NeoSelenium
'''

from NeoSelenium import initEngine
from NeoSelenium import deInitEngine

#一个简单的查询测试
def neohope_search_test():
    try:
        #myEngine = initEngine('ie32')
        #myEngine = initEngine('ie64')
        #myEngine = initEngine('chrome')
        myEngine = initEngine('ff32')
        #myEngine = initEngine('ff64')

        base_url = "https://www.neohope.org"
        myEngine.get(base_url + "/")
        myEngine.find_element_by_name("s").clear()
        myEngine.find_element_by_name("s").send_keys("Metabase")
        myEngine.find_element_by_css_selector("button.search-submit").click()
	myEngine.implicitly_wait(1000)
	#print(myEngine.find_elements_by_xpath("//div[@id='content']/article"))
	queryResultLenght = len(myEngine.find_elements_by_xpath("//div[@id='content']/article"))
	#print(queryResultLenght)
	#应该是1但现在是10
        assert queryResultLenght==1
    finally:
        #deInitEngine(myEngine)
	print("test end")

#start here
neohope_search_test()

5、运行脚本

PS:
IE驱动抛出异常解决方案

"Unexpected error launching Internet Explorer. Protected Mode settings are not the same for all zones. Enable Protected Mode must be set to the same value (enabled or disabled) for all zones."

解决方法:

     
打开IE->Internet options->Security->Enable Protected Mode
这个框,在四个域下面要是一致的(要么都勾上,要么都不勾上)

Selenium入门01

1、常用模块介绍

Selenium IDE 是一个FF插件,用于录制测试用例并重新运行。可以保持为html脚本,也可以导出为各种语言的单元测试
Selenium Html Runner 可以直接运行Selenium IDE导出的html脚本
Selenium Standalone Server 有三种运行模式(standalone、hub、node),后两种用于grid
Selenium Remote Control Selenium RC, Selenium1采用代理网站及JS注入的方式,达到操控网页的目的,在Selenium3已经取消支持
Browser Drivers Selenium2为在不修改网页的情况下,达到操控网页的无敌,利用了各种浏览器的API,达到操控浏览器的目的
Remote Web Drivers 远程运行测试(与Selenium Standalone Server或Selenium Standalone Hub进行交互时使用)
Selenium GRID 用于同时对多个操作系统的多种浏览器进行自动化测试(包括一个HUB,和多个NODE)
Language Binding 各种开发语言包

2、Selenium IDE
2.1、FF下载插件并重启
https://addons.mozilla.org/en-US/firefox/addon/selenium-ide/
2.2、录制插件

用FF打开你要测试的网站
打开Selenium IDE
Tools->Selenium IDE
输入BaseURL,点击右侧小红点儿,开始录制
在页面上进行操作(最好不要切换页面)

2.3、保存TestSuit及TestCase

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Strict//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd">
<html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en">
<head>
  <meta content="text/html; charset=UTF-8" http-equiv="content-type" />
  <title>Test Suite</title>
</head>
<body>
<table id="suiteTable" cellpadding="1" cellspacing="1" border="1" class="selenium"><tbody>
<tr><td><b>Test Suite</b></td></tr>
<tr><td><a href="neohope.case.search.html">neohope_search_testcase001</a></td></tr>
</tbody></table>
</body>
</html>

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Strict//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd">
<html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en">
<head profile="http://selenium-ide.openqa.org/profiles/test-case">
<meta http-equiv="Content-Type" content="text/html; charset=UTF-8" />
<link rel="selenium.base" href="https://www.neohope.org/" />
<title>neohope_search_testcase001</title>
</head>
<body>
<table cellpadding="1" cellspacing="1" border="1">
<thead>
<tr><td rowspan="1" colspan="3">neohope_search_testcase001</td></tr>
</thead><tbody>
<tr>
	<td>open</td>
	<td>/</td>
	<td></td>
</tr>
<tr>
	<td>type</td>
	<td>name=s</td>
	<td>Metabase</td>
</tr>
<tr>
	<td>clickAndWait</td>
	<td>css=button.search-submit</td>
	<td></td>
</tr>
<tr>
	<td>assertXpathCount</td>
	<td>//article</td>
	<td>1</td>
</tr>

</tbody></table>
</body>
</html>

2.4、导出python脚本
Selenium IDE->File->Export Test Case As…->Python2/WebDriver

3、用selenium-html-runner运行
3.1、下载并配置jdk
3.2、下载selenium-html-runner-3.0.1.jar
3.3、在selenium网站下载对应浏览器的driver
http://www.seleniumhq.org/download/
3.4、运行

set JAVA_HOME=C:\NeoLanguages\Java\JDK\jdk_x86_1.8.0_77
set PATH=%JAVA_HOME%\bin;%PATH%;C:\NeoTest\TestSelenium\trunk\bin\x86\;
set webdriver.gecko.driver=C:\NeoTest\TestSelenium\trunk\bin\x86\geckodriver.exe

java -jar ../lib/selenium-html-runner-3.0.1.jar -htmlSuite *firefox "https://www.neohope.org/" "../testcases/neohope.suit" "../testcases/neohope.result"
PAUSE

测试工具推荐

测试工具 适用问题 编程语言
JUnit 单元测试 Java
NUnit 单元测试 C#
PyUnit 单元测试 Python
TestUnit 单元测试 Ruby
QTP 自动化测试(BS、CS) VBS
Selenium 自动化测试(BS) Python、Ruby、Java、C#、JS
RobotFramework 自动化测试(BS) Python
Watir 自动化测试(BS) Ruby
LR 性能测试(BS、CS) C
Jmeter 接口测试、性能测试(BS) Java
SoapUI/Ready! API 接口测试、性能测试(BS) Java

其中:
1、老牌测试工具LR和QTP等,对新版本浏览器支持欠佳
2、暂时没找到很好的CS端,自动化测试工具
3、对于测试人员来说,脚本的学习曲线没有那么陡峭,建议Python或Ruby

Redash环境搭建(Ubuntu)

1、下载安装脚本

wget -O bootstrap.sh https://raw.githubusercontent.com/getredash/redash/master/setup/ubuntu/bootstrap.sh

2、运行脚本

chmod +x bootstrap.sh
sudo ./bootstrap.sh

3、脚本执行成功后,直接访问nginx就好了
http://ip:80
实际上是代理了这个网站
http://localhost:5000

4、常见问题
在执行过程中,遇到下载失败的情况,就直接把文件下载到本地,改一下路径,重新运行脚本就好了
我在运行脚本的过程中,遇到了缺少schema的提示,删除了数据库redash及用户redash,重新运行脚本就好了

Superset环境搭建(Ubuntu)

1、安装依赖包

sudo apt-get install build-essential libssl-dev libffi-dev python-dev python-pip libsasl2-dev libldap2-dev
pip install virtualenv

2、使用虚拟环境

#新建沙盒
virtualenv supersetenv
#进入沙盒
source bin/activate

3、安装

#升级安装工具,安装服务
pip install --upgrade setuptools pip
pip install superset
#新增管理员用户
fabmanager create-admin --app superset
#重置管理员密码
#fabmanager reset-password admin --app superset
#升级数据库
superset db upgrade
#加载测试数据
superset load_examples
#初始化
superset init
#运行Server
superset runserver

4、此时,只需要访问http://ip:8088就可以登陆了

5、安装驱动

#mysql
apt-get install libmysqlclient-dev
pip install mysqlclient
#oracle
#pip install cx_Oracle
#mssql
#pip install pymssql

6、关闭

#Ctrl+C关闭Server
#退出沙盒
deactivate
#删除沙盒
#rmvirtualenv supersetenv

7、升级

#进入沙盒
source bin/activate
#升级Server
pip install superset --upgrade
#升级DB
superset db upgrade
#初始化
superset init
#退出沙盒
deactivate

Spark环境搭建05

这里主要说一下Spark的SQL操作,如何从mysql导入数据。

#拷贝了mysql的驱动到jar文件下面
#建立DF
val jdbcDF = spark.read.format("jdbc").option("url", "jdbc:mysql://localhost:3307").option("dbtable", "hive.TBLS").option("user", "hive").option("password", "hive").load()

#schema
jdbcDF.schema

#count
jdbcDF.count()

#show
jdbcDF.show()

Spark环境搭建04

这里主要说一下Spark的SQL操作。

1、dataframe操作数据

#加载json数据
val df = spark.read.json("/usr/hadoop/person.json")
#加载CSV数据
#val df = spark.read.csv("/usr/hadoop/person.csv")

#查询前20行
df.show()

#查看结构
df.printSchema()

#选择一列
df.select("NAME").show()

#按条件过滤行
df.filter($"BALANCE_COST" < 10 && $"BALANCE_COST" > 1).show()

#分组统计
df.groupBy("SEX_CODE").count().show()

2、sql操作数据

#创建视图
df.createOrReplaceTempView("person")

#查看数据
spark.sql("SELECT * FROM person").show()

#统计数据
spark.sql("SELECT * FROM person").count()

#带条件选择
spark.sql("SELECT * FROM person WHERE BALANCE_COST<10 and BALANCE_COST>1 order by BALANCE_COST").show()

3、转为DS

#转为Dataset
case class PERSONC(PATIENT_NO : String,NAME : String,SEX_CODE : String,BIRTHDATE : String,BALANCE_CODE : String)
var personDS = spark.read.json("/usr/hadoop/person.json").as[PERSONC]

4、sql与map reduce混用

personDS.select("BALANCE_COST").map(row=>if(row(0)==null) 0.0 else (row(0)+"").toDouble).reduce((a,b)=>if(a>b) a else b)

spark.sql("select BALANCE_COST from person").map(row=>if(row(0)==null) 0.0 else (row(0)+"").toDouble).reduce((a,b)=>if(a>b) a else b)

5、数据映射为Class对象

val personRDD = spark.sparkContext.textFile("/usr/hadoop/person.txt")
val persons = personRDD.map(_.split(",")).map(attributes => Person(attributes(0), attributes(1), attributes(2), attributes(3), attributes(4).toDouble))

6、自定义schema

import org.apache.spark.sql.Row
import org.apache.spark.sql.types._

#加载数据
val personRDD = spark.sparkContext.textFile("/usr/hadoop/person.txt")
#转为org.apache.spark.sql.Row
val rowRDD = personRDD.map(_.split(",")).map(attributes => Row(attributes(0), attributes(1), attributes(2), attributes(3), attributes(4).replace("\"","").toDouble))
#定义新的Schema
val personSchema = StructType(List(StructField("PatientNum",StringType,nullable = true), StructField("Name",StringType,nullable = true), StructField("SexCode",StringType,nullable = true), StructField("BirthDate",StringType,nullable = true), StructField("BalanceCode",DoubleType,nullable = true)))
#建立新的DF
val personDF = spark.createDataFrame(rowRDD, personSchema)
#使用DF
personDF.select("PatientNum").show()