Testing SAP: Modern Testing Methodologies: Ethan Jewett
Uploaded by
bharikrishnan17701Testing SAP: Modern Testing Methodologies: Ethan Jewett
Uploaded by
bharikrishnan17701Testing SAP: Modern Testing Methodologies
Ethan Jewett
But first!
Whats the problem?
Enterprise project trade-offs
Value
Budget
Risk
Duration
Software development trade-offs
Features
Budget
Quality
Time
Client requests
Build Faster And produce more Value Cheaper with less Risk
Remember
Value
Budget
Risk
Duration
Are there hidden assumptions?
Yes. Methodology People and Culture Technology
Any other assumptions?
Project Preparation Business Blueprint Project Realization
Final Preparation Go-live and Support
Technical Design Technical Build
Write and run unit test
Test Fix
Transport
Go Live
Live Support
Write test scripts
Integration & Regression Test
Training
The topic
Over the next hour and a half, well talk about how testing methodologies, techniques, and technologies can help us Change these underlying assumptions. Enable modern project methodologies. Control the basic project trade-off.
The focus
Project Preparation Business Blueprint Project Realization
Final Preparation Go-live and Support
Technical Design Technical Build
Write and run unit test
Test Fix
Transport
Go Live
Live Support
Write test scripts
Integration & Regression Test
Training
The goal
More value Faster With less risk
(and more fun)
For example
V-Model methodology
http://en.wikipedia.org/wiki/File:V-model.JPG
More parallel, less repetition
http://en.wikipedia.org/wiki/File:V-model.JPG
Faster
http://en.wikipedia.org/wiki/File:V-model.JPG
More iterative
http://en.wikipedia.org/wiki/File:V-model.JPG
How do we get there?
Modern methodology
Modern quality assurance and testing
Leading practice in custom development, open source, and agile projects
Test driven development (TDD), behavior driven development (BDD), readable tests, executable requirements, continuous integration
Overview of the SAP test automation technologies Road we're going to take through testing SAP
Leading Practice
Context Questions and problems Approaches Tools
Leading Practice
Context Questions and problems Approaches Tools
Context
The context of modern software development is above all open, agile and decentralized. Open source projects are usually the furthest along this path, and as such these projects use (or are forced to invent) tools that fit their processes.
Openness
Requirements, specifications, issues reports, test cases, source code, and project tools and metrics are readily open and available to all participants in the project. (Not necessarily the same as open source.)
Open Collaboration within Corporations Using Software Forges http://www.riehle.org/publications/2009/open-collaboration-within-corporations-using-software-forges
Agility
The values of the Agile Manifesto:
Individuals and interactions over processes and tools Working software over comprehensive documentation Customer collaboration over contract negotiation Responding to change over following a plan
http://agilemanifesto.org/
Decentralization
Open-source projects are decentralized by nature, but most organizations today have some element of decentralization Challenges:
Quality control Communication Maintaining commitment
Leading Practice
Context Questions and problems Approaches Tools
Questions and problems
How do we enforce disciplined testing? Do we use unit, functional, integration, or acceptance testing? Or all of them? How do we ensure that the build complies with the tests?
How do we track 100s or 1000s of bugs (including duplicates)? How do we make tests relevant? How does testing integrate with issue tracking? Manual or automatic testing?
How do we determine the cause of test failure?
Leading Practice
Context Questions and problems Approaches Tools
Approaches
Test Coverage Test Automation Test Driven Development Behavior Driven Development Spec as Test (the test/spec convergence) Exploratory Testing Continuous Integration
Test coverage
The percentage of code or development that is tested. In code, this might be measured by determining if every branch of code can result in a failing test condition (Branch coverage)
Wikipedia - http://en.wikipedia.org/wiki/Code_coverage
Test automation
Accepted as gospel modern dev communities
Regardless of how good they are at testing, accept that they should automate as much as possible Can lead to ignoring non-automatable testing
In the SAP world we havent even accepted that full test coverage is desirable, much less automation
Test automation pushback http://railspikes.com/2008/7/11/testing-is-overrated http://michaelfeathers.typepad.com/michael_feathers_blog/2008/06/the-flawedtheo.html Automated testing story on SDN (never completed) https://www.sdn.sap.com/irj/scn/weblogs?blog=/pub/wlg/4103
A note on the following slides
Im using bowling as an example because Im copying from the internet. I dont know much about bowling, so the examples are probably wrong. My assumption is that a game is usually 10 frames of two balls per frame. If the last frame is a spare or a strike, you get at most two additional frames. Or something like that.
Test Driven Development (TDD)
Build
Design
Test
Technical specification
Design
Write tests based on specs
Run tests Failure!
Build
Write code to address test failures
Run tests
Design
Test
Test
TDD Example
require test/unit require test/unit/assertions require bowling
class TC_Bowling < Test::Unit::TestCase def setup bowling = Bowling.new end def gutter_game 20.times { bowling.hit(0) } assert bowling.score == 0 end end
Behavior Driven Development (BDD)
Focus on the external behavior of the code Abstract away implementation details Additionally, BDD libraries tend to codify the best practices of unit testing and TDD
BDD example
require spec require bowling
describe Bowling do
it "should score 0 for gutter game" do
bowling = Bowling.new 20.times { bowling.hit(0) } bowling.score.should == 0 end end
Spec as Test or writing features
Recently, the Ruby community has begun developing testing frameworks that are even closer to natural language.
Cucumber
These frameworks wrap TDD and BDD frameworks and allow for business-writable, business-readable, executable test scripts.
Feature example
Feature (visible to the user) Implementation (not visible to user)
require spec/expectations require bowling
Given /I have knocked over (\d+) pins per ball/ do |pins| @pins_per_ball = pins end Given /I have bowled (\d+) balls/ do |balls| @bowling = Bowling.new balls.times { @bowling.hit( @pins_per_ball ) } end Then /I should have (\d+) points/ do |points| @bowling.score.should == points end
Scenario: Gutter game Given I have bowled 20 balls and I have knocked over 0 pins per ball When I check the score Then I should have 0 points
Feature example cont.
Note that we can now implement many more tests with no more code/implementation:
Scenario: Perfect game Given I have bowled 12 balls and I have knocked over 10 pins per ball When I check the score Then I should have 300 points Scenario: Bad game Given I have bowled 20 balls and I have knocked over 2 pins per ball When I check the score Then I should have 40 points Scenario: Lots of spares Given I have bowled 24 balls and I have knocked over 5 pins per ball When I check the score Then I should have ??? Points Or maybe I need something a little different.... Scenario: Too long a game Given I have bowled 30 balls and I have knocked over 0 pins per ball Then I should receive an error
Side-by-side
TDD
require test/unit require test/unit/assertions require bowling class TC_Bowling < Test::Unit::TestCase
BDD
require spec/expectations require bowling describe Bowling do
Spec as Test
Scenario: Gutter game Given I have bowled 20 balls and I have knocked over 0 pins per ball
it "should score 0 for gutter game" do
def setup bowling = Bowling.new end def gutter_game 20.times { bowling.hit(0) } assert bowling.score == 0 end end bowling = Bowling.new 20.times { bowling.hit(0) } bowling.score.should == 0 end end
When I view the score
Then I should have 0 points
TDD, BDD, and Test-as-Spec References
Test Driven Design/Development
http://en.wikipedia.org/wiki/Testdriven_development http://www.agiledata.org/essays/tdd.html
Behavior Driven Development
http://behaviour-driven.org/
Spec as test
Cucumber - http://cukes.info/ Rspec - http://rspec.info/ http://www.pragprog.com/titles/achbd/the-rspecbook
Exploratory Testing
The practice of trying to break things
Career security for klutzes
Exploratory testing appears informal, but can be structured and is a very important aspect of software testing.
Probably the most neglected form of testing in open source projects
http://www.kohl.ca/blog/archives/000185.html http://www.kohl.ca/articles/ExploratoryTesting_MusicofInvestigation.pdf
Continuous Integration
The practice of automating not only your tests but your full commit-build-test cycle
1. A commit new or changed code 2. Triggers a full system build 3. And an execution of the entire test suite
Cruisecontrol.rb http://rubyforge.org/projects/cruisecontrolrb http://cruisecontrolrb.thoughtworks.com/ Hudson https://hudson.dev.java.net/ http://www.softwarebloat.com/2008/11/19/continuous-integration-blueprints-howto-build-an-army-of-killer-robots-with-hudson-and-cucumber
Leading Practice
Context Questions and problems Approaches Tools
Tools
Source version control (svn, git) Testing libraries (rspec, cucumber, junit, ABAPunit, jspec) Continuous Integration tools (cruisecontrol.rb, Hudson) Issue tracking (Sourceforge, trac, Lighthouse, Google Code) Agile methodologies (Scrum, XP, etc.)
Modern methodology
Modern quality assurance and testing
Leading practice in custom development, open source, and agile projects
Test driven development (TDD), behavior driven development (BDD), readable tests, executable requirements, continuous integration
Overview of the SAP test automation technologies Road we're going to take through testing SAP
The SAP World
Manual Unit testing
ABAP Unit Java
Functional testing
eCATT
ABAP Unit
Modeled on Java Unit Excellent (if a bit unwieldy) for traditional unit test automation or classes
Works well for TDD
ABAP Objects (object-oriented ABAP)
TDD Demo ZBOWLING & ZBOWLING_TEST
http://help.sap.com/saphelp_nw70ehp1/helpdata/en/a2/8a1b602e85 8645b8aac1559b638ea4/frameset.htm
Non-SAP
We can use just about anything via RFCs For Web Dynpros, BSPs, or Portal applications, there are a lot of options using web-drivers that can automatically drive browsers Address in depth in future sessions
Modern methodology
Modern quality assurance and testing
Leading practice in custom development, open source, and agile projects
Test driven development (TDD), behavior driven development (BDD), readable tests, executable requirements, continuous integration
Overview of the SAP test automation technologies Road we're going to take through testing SAP
The Road to the Future
Developing techniques to support agile, open, decentralized development in an SAP landscape Using SAP tools and 3rd party tools Shorter, more focused and hands-on sessions
Start with this
http://en.wikipedia.org/wiki/File:V-model.JPG
Get more parallel, less repetitive
Spec = Test
http://en.wikipedia.org/wiki/File:V-model.JPG
Get faster
Test Automation
http://en.wikipedia.org/wiki/File:V-model.JPG
Get iterative
Continuous
Integration
http://en.wikipedia.org/wiki/File:V-model.JPG
And end up here (or somewhere similar)
http://en.wikipedia.org/wiki/File:Scrum_process.svg
Thanks
General references for inclusion
Open Collaboration within Corporations Using Software Forges - http://www.riehle.org/publications/2009/open-collaboration-within-corporations-using-software-forges/ Continuous integration tools
http://rubyforge.org/projects/cruisecontrolrb - http://cruisecontrolrb.thoughtworks.com/ https://hudson.dev.java.net/ http://www.softwarebloat.com/2008/11/19/continuous-integration-blueprints-how-to-build-an-army-of-killer-robots-with-hudson-and-cucumber http://www.kohl.ca/blog/archives/000185.html http://www.kohl.ca/articles/ExploratoryTesting_MusicofInvestigation.pdf http://www.kaner.com/pdfs/TheOngoingRevolution.pdf Spotlight on ABAP Unit Part 1 - https://www.sdn.sap.com/irj/scn/weblogs?blog=/pub/wlg/1088 http://dannorth.net/2007/02/monkey-business-value
Exploratory testing
The ongoing revolution in software testing ABAP Unit Load testing Joel Spolsky 12 steps to better code - http://www.joelonsoftware.com/articles/fog0000000043.html Automated testing story on SDN - https://www.sdn.sap.com/irj/scn/weblogs?blog=/pub/wlg/4103 JUnit - http://www.junit.org/ Open Source
http://www.paulgraham.com/opensource.html (What business can learn from open source) http://wtr.rubyforge.org/ Cucumber - http://cukes.info/ Rspec - http://rspec.info/ http://www.pragprog.com/titles/achbd/the-rspec-book http://www.beteoblog.com/beteo-alm-miniguides/software-quality/ http://raa.ruby-lang.org/project/saprfc/ Integration Tests in ABAP Development und Tool Support - https://www.sdn.sap.com/irj/scn/weblogs?blog=/pub/wlg/7131 XSLT - https://www.sdn.sap.com/irj/scn/weblogs?blog=/pub/wlg/12173 http://railspikes.com/2008/7/11/testing-is-overrated http://michaelfeathers.typepad.com/michael_feathers_blog/2008/06/the-flawed-theo.html
Watir
Spec as test
SAP testing
