Software Testing

You will learn how to test your code to make sure it works how it should.

Why Test?

Testing is a crucial phase in the software development lifecycle. It ensures that the software meets the required standards of quality, reliability, and performance. Proper testing can prevent bugs, reduce development costs, and improve user satisfaction. In this blog post, we will explore various tips and best practices for effective software testing.

Understand the Requirements

Before starting the testing process, it is essential to have a clear understanding of the software requirements. This includes:

• Functional Requirements: What the software is supposed to do.
• Non-Functional Requirements: Performance, usability, security, and other aspects.

Understanding these requirements helps in designing relevant test cases and ensures comprehensive coverage.

Plan Your Tests

A well-structured test plan is the foundation of effective testing. It should include:

• Scope: Define what will and will not be tested.
• Objectives: Outline the goals of the testing process.
• Resources: Identify the necessary tools, environments, and personnel.
• Schedule: Set timelines for different testing phases.
• Risk Management: Anticipate potential risks and plan mitigation strategies.

A detailed test plan ensures that the testing process is organized and systematic.

Write Clear and Concise Test Cases

Test cases should be clear, concise, and easy to understand. They should include:

• Test Steps: Detailed steps to execute the test.
• Expected Results: The expected outcome of the test.
• Test Data: Any data required for the test.

Clear test cases ensure that the tests can be easily executed and repeated by different testers.

Automate Where Possible

Automated testing can save time and improve accuracy. Consider automating:

• Regression Tests: To ensure that new changes do not break existing functionality.
• Performance Tests: To check how the application performs under load.
• Unit Tests: To verify the functionality of individual components.

Tools like Selenium, JUnit, and TestNG can help automate various types of tests.

Perform Manual Testing

While automation is valuable, manual testing is also crucial for:

• Exploratory Testing: To discover unexpected issues through ad-hoc testing.
• Usability Testing: To assess the user experience and interface.
• Ad-Hoc Testing: To test random aspects of the application based on experience and intuition.

Manual testing helps catch issues that automated tests might miss.

Test Early and Often

Implement testing early in the development process and continue it throughout the lifecycle. This includes:

• Unit Testing: Testing individual components during development.
• Integration Testing: Testing the interaction between components as they are integrated.
• System Testing: Testing the complete system as a whole.
• Acceptance Testing: Ensuring the system meets the business requirements before release.

Continuous testing helps identify and fix issues promptly, reducing the cost and effort required to resolve them later.

Use a Test Management Tool

Test management tools can help organize and manage the testing process. They offer features like:

• Test Case Management: Creating, storing, and managing test cases.
• Test Execution: Tracking the execution of tests and logging results.
• Defect Management: Recording and managing bugs and issues.
• Reporting: Generating reports on test coverage, progress, and results.

Tools like Jira, TestRail, and Zephyr streamline the testing process and improve collaboration.

Keep Communication Open

Effective communication among team members is essential for successful testing. This includes:

• Regular Meetings: Hold regular meetings to discuss progress, issues, and plans.
• Clear Reporting: Provide clear and detailed reports on test results and issues.
• Collaboration: Foster collaboration between developers, testers, and other stakeholders.

Open communication helps ensure that everyone is on the same page and working towards common goals.

Focus on Security Testing

With the increasing number of security threats, it is crucial to focus on security testing. This includes:

• Vulnerability Scanning: Identifying potential vulnerabilities in the application.
• Penetration Testing: Simulating attacks to find security weaknesses.
• Security Code Review: Reviewing the code for security issues.

Security testing helps protect the application from threats and ensures compliance with security standards.

Learn from Defects

Every defect found during testing is an opportunity to learn and improve. Analyze defects to:

• Identify Root Causes: Understand why the defect occurred.
• Improve Processes: Adjust development and testing processes to prevent similar issues in the future.
• Enhance Test Cases: Update test cases to cover new scenarios and prevent regression.

Learning from defects helps improve the overall quality of the software and the effectiveness of the testing process.

In a Nutshell

Effective software testing is a critical component of successful software development. By understanding requirements, planning tests, writing clear test cases, automating where possible, performing manual testing, testing early and often, using test management tools, keeping communication open, focusing on security testing, and learning from defects, you can ensure that your software is robust, reliable, and ready for release. Remember, investing time and effort in thorough testing ultimately leads to higher quality software and greater user satisfaction. Testing allow you to make sure your code is working the way you want it to. If there are any changes to your code, tests should fail.

What you want

import unittest

def sum(x, y):
  return x + y

class TestAdd(unittest.TestCase):
  def test_add_positive(self):
    """Tests that adding two positive numbers works."""
    result = sum(2, 3)
    self.assertEqual(result, 5)

if __name__ == '__main__':
  unittest.main()

More Tests

import unittest

def add(x, y):
  """Adds two numbers together.

  Args:
      x: The first number.
      y: The second number.

  Returns:
      The sum of x and y.
  """
  return x + y

class TestAdd(unittest.TestCase):

  def test_add_positive(self):
    """Tests that adding two positive numbers works."""
    result = add(2, 3)
    self.assertEqual(result, 5)

  def test_add_negative(self):
    """Tests that adding two negative numbers works."""
    result = add(-2, -5)
    self.assertEqual(result, -7)

  def test_add_zero(self):
    """Tests that adding zero to a number works."""
    result = add(0, 10)
    self.assertEqual(result, 10)

  # You can add more test cases here for different scenarios

if __name__ == '__main__':
  unittest.main()

Work Backwords

Code Testing

coverage

Absolutely, let's break down these common code testing types:

Unit Testing

• Focuses on individual units of code, typically functions or classes.
• Tests ensure the unit performs as designed, handling expected inputs and producing correct outputs.
• Developers often write unit tests alongside the code itself.

Details

accounts.py

class Currency():
    def __init__(self,amount):
        self.amount = amount
        self._currency = "${:,.2f}".format(self.amount)
    
    def as_currency(self):
        return self._currency
        
class Account():
    all = []
    archived = []
    def __init__(self,name=""):
        self.name = name
        self.__balance = 0 # private (only accesed by methods). Doesn't allow self.balance
        self.__closed = False
        Account.all.append(self)
        
    @classmethod
    def get_iterative_totals(cls):
        total = 0
        for i in cls.all:
            total += i.__balance
        return Currency(total).as_currency()
    
    @classmethod
    def get_recursive_totals(cls, accounts_array):
        if len(accounts_array) == 0:
            return 0
        else:
            total = accounts_array.pop().__balance
            return total + cls.get_recursive_totals(accounts_array)
    
    def __add__(self, other):
        total = self.get_balance() + other.get_balance()
        return total

    def __repr__(self) -> str:
        return self.name
    
    def __str__(self) -> str:
        return self.name

    def close(self):
        self.__closed = True
        Account.archived.append(self)
    
    @property
    def check_balance(self):
        return f'The balance of {self.name} is {self.__balance}'
    
    def get_balance(self):
        return self.__balance
    
    def withdraw_cash(self, amount):
        self.__balance -= amount
        print(f'You withdrew ${amount} from your \'{self.name}\' account. {self.check_balance()}')
    
    def deposit(self,amount):
        self.__balance += amount
        print(f'You deposited ${amount} into your \'{self.name}\' account. {self.check_balance()}')
    
    def transfer_money(self,amount,destination):
        self.__balance -= amount
        destination.__balance += amount
        Transaction(self,destination)
        Notification(f'Transaction from {self.name} to {destination.name} is complete')
        print(f'You transfered ${amount} from \'{self}\' to \'{destination}\'')

class Notification():
    def __init__(self, title):
        self.title = title
        print(self.title)

class Transaction():
    all = []
    def __init__(self,from_account,to_account):
        self.from_account = from_account
        self.to_account = to_account
        Transaction.all.append([from_account,to_account])

# Create new accounts
acc1 = Account('BANKOZK')
acc2 = Account('BAM')
acc2 = Account('CAPITALONE')

# Transfers
acc1.transfer_money(34.00,acc2)
acc2.transfer_money(24.00,acc1)

# Withdrawl
acc2.withdraw_cash(2)
acc2.withdraw_cash(9)

# Deposits
acc2.deposit(30)
acc1.deposit(30)

# Inquiry
print(acc2.check_balance())

# Other
print('Iterative Totals')
print(Account.get_iterative_totals())
print('Recursive Totals')
print(Account.get_recursive_totals(Account.all))

# Get all transactions
print(Transaction.all)
for A,B in Transaction.all:
    print(A.check_balance(),B.check_balance())

# Overrides
print("Adding Accounts")
print(acc1+acc2)

#
acc1.close()
print(Account.archived)

test_accounts.py

import unittest
from accounts import Account,Notification,Transaction

class testAccounts(unittest.TestCase):
    def setUp(self):
        self.account = Account('BANKOZK')
    def test_add_account(self):
   	    self.assertEqual(self.account.check_balance(),'The balance of BANKOZK is 0')
   	    self.assertEqual(len(Account.all),1)

if __name__ == "__main__":
    unittest.main()

Integration Testing

• Tests how multiple units of code work together.
• Ensures different modules communicate and exchange data correctly.
• Often involves testing interactions between classes or functions within a single component.

Details

from unittest.mock import patch

# Replace these with your actual classes
from banking_app import Account, Transaction, Notification

def test_deposit_triggers_notification(mocker):
  """Tests if depositing money triggers a notification."""
  # Mock the notification functionality
  mock_send_notification = mocker.patch.object(Notification, "send")

  # Create an account
  account = Account(1234, 100)

  # Deposit some amount
  amount = 50
  transaction = Transaction(account.number, amount)
  account.deposit(transaction)

  # Assert notification is sent with correct message
  mock_send_notification.assert_called_once_with(f"Deposited ${amount}. New balance: ${account.balance}")

def test_withdrawal_triggers_notification(mocker):
  """Tests if withdrawing money triggers a notification."""
  # Mock the notification functionality
  mock_send_notification = mocker.patch.object(Notification, "send")

  # Create an account with sufficient balance
  account = Account(5678, 200)

  # Withdraw some amount
  amount = 100
  transaction = Transaction(account.number, -amount)
  account.withdraw(transaction)

  # Assert notification is sent with correct message
  mock_send_notification.assert_called_once_with(f"Withdrew ${amount}. New balance: ${account.balance}")

# Test case for insufficient funds scenario (optional)
def test_withdrawal_insufficient_funds(mocker):
  """Tests if withdrawing with insufficient funds triggers notification."""
  # Implement logic similar to previous tests with insufficient balance

# You can add more test cases for different functionalities

if __name__ == "__main__":
  unittest.main()

Functional Tests

• Verify the software's functionalities from the user's perspective.
• Tests ensure features behave as specified in the requirements document.
• Focuses on what the system does, not how it does it.

Details

from banking_app import Account, Transaction, Notification

def test_deposit_sends_notification(capfd):
  """Tests if depositing money triggers a notification."""
  # Create an account
  account = Account(1234, 100)

  # Deposit some amount
  amount = 50
  transaction = Transaction(account.number, amount)
  account.deposit(transaction)

  # Capture output from print statements (or notification logic)
  output = capfd.readouterr()[0].strip()

  # Assert notification message is present in the output
  assert f"Deposited ${amount}. New balance: ${account.balance}" in output

def test_withdrawal_sends_notification(capfd):
  """Tests if withdrawing money triggers a notification."""
  # Create an account with sufficient balance
  account = Account(5678, 200)

  # Withdraw some amount
  amount = 100
  transaction = Transaction(account.number, -amount)
  account.withdraw(transaction)

  # Capture output from print statements (or notification logic)
  output = capfd.readouterr()[0].strip()

  # Assert notification message is present in the output
  assert f"Withdrew ${amount}. New balance: ${account.balance}" in output

# Test case for insufficient funds scenario (optional)
def test_withdrawal_insufficient_funds(capfd):
  """Tests if insufficient funds are handled correctly."""
  # Implement logic similar to previous tests with insufficient balance

  # Assert appropriate error message is displayed (optional)
  # assert "Insufficient funds" in output

if __name__ == "__main__":
  import pytest
  pytest.main()

(End-to-End, E2E Testing)

• Simulates real user workflows and tests entire application flows.
• Involves multiple components and integrations working together.
• Ensures the overall user experience functions as intended from start to finish.

Details

import json
from unittest import mock

# Replace these with your actual classes
from banking_app import Account, Transaction, Notification, create_app

def test_end_to_end_deposit_notification(client):
  """Tests deposit functionality with notification via Flask app."""
  # Simulate user deposit via a POST request
  account_number = 1234
  deposit_amount = 50
  data = {"account_number": account_number, "amount": deposit_amount}
  response = client.post('/deposit', json=data)

  # Assert successful deposit response
  assert response.status_code == 200
  json_response = response.get_json()
  assert json_response["message"] == f"Deposited ${deposit_amount}. New balance: ${json_response['balance']}"

def test_end_to_end_withdrawal_notification(client):
  """Tests withdrawal functionality with notification via Flask app."""
  # Simulate user creating an account with sufficient balance
  initial_balance = 200
  account_number = create_account(initial_balance)  # Replace with account creation logic

  # Simulate user withdrawal via a POST request
  withdrawal_amount = 100
  data = {"account_number": account_number, "amount": withdrawal_amount}
  response = client.post('/withdraw', json=data)

  # Assert successful withdrawal response
  assert response.status_code == 200
  json_response = response.get_json()
  assert json_response["message"] == f"Withdrew ${withdrawal_amount}. New balance: ${json_response['balance']}"

# Helper function to create an account (replace with actual logic)
def create_account(initial_balance):
  # Implement logic to create an account with initial balance and return account number
  pass

# You can add more test cases for different functionalities

def test_insufficient_funds_withdrawal(client):
  """Tests insufficient funds scenario during withdrawal."""
  # Simulate insufficient balance scenario (implement logic in POST request)
  # Assert appropriate error response (e.g., 400 Bad Request)

if __name__ == "__main__":
  app = create_app()
  with app.test_client() as client:
    test_end_to_end_deposit_notification(client)
    test_end_to_end_withdrawal_notification(client)
    test_insufficient_funds_withdrawal(client)

Acceptance Testing

• Performed by the customer or end-users to validate if the software meets their needs.
• Confirms the system fulfills the acceptance criteria agreed upon before development.
• Crucial step before releasing the software to ensure it meets user expectations.

Details

from selenium import webdriver
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC

def test_deposit_with_notification(driver):
  # Login to the application (replace with actual login logic)
  driver.get("http://localhost:5000/login")
  username_field = driver.find_element(By.ID, "username")
  username_field.send_keys("test_user")
  password_field = driver.find_element(By.ID, "password")
  password_field.send_keys("password")
  driver.find_element(By.ID, "login_button").click()

  # Navigate to deposit page and enter amount
  driver.get("http://localhost:5000/deposit")
  deposit_amount_field = driver.find_element(By.ID, "deposit_amount")
  deposit_amount_field.send_keys(50)
  driver.find_element(By.ID, "deposit_button").click()

  # Wait for confirmation message and assert its presence
  confirmation_text = WebDriverWait(driver, 10).until(
      EC.presence_of_element_located((By.ID, "confirmation_message"))
  ).text
  assert "Deposited $50" in confirmation_text

# Similar tests can be written for withdrawal, account creation, etc.

Performance Testing

• Evaluates how the software behaves under load.
• Measures factors like speed, stability, and scalability under various conditions.
• Helps identify performance bottlenecks and ensure the system can handle expected usage.

Details

from locust import HttpUser, task, between

class BankingUser(HttpUser):
  wait_time = between(1, 2)  # Simulate user think time between actions

  @task
  def deposit(self):
    account_number = 1234
    amount = 50
    data = {"account_number": account_number, "amount": amount}
    response = self.client.post("/deposit", json=data)
    assert response.status_code == 200

  @task
  def withdraw(self):
    account_number = 5678
    amount = 20
    data = {"account_number": account_number, "amount": amount}
    response = self.client.post("/withdraw", json=data)
    assert response.status_code == 200

def test_load_performance(client):
  # Configure number of virtual users and spawn rate
  user_count = 100
  spawn_rate = 10  # Spawn 10 users per second

  with client as l:
    l.spawn(BankingUser, weight=user_count, spawn_rate=spawn_rate)
    l.start()
    l.wait()

  # Analyze test results from Locust web interface

Smoke Testing

• Basic tests performed after a new build or code deployment.
• Aims to quickly identify critical issues that prevent core functionalities from working.
• Ensures the build is stable enough for further testing before investing time in more complex tests.

Smoke testing for a Flask banking application is a quick and basic sanity check to ensure the core functionalities work before proceeding with further testing. Here's how you can perform smoke testing:

Manually Testing Endpoints:

1. Start the Flask application: Run your Flask application in development mode to allow for easier testing.
2. Test core functionalities:
   • Use tools like Postman or curl to send manual requests to your application's endpoints.
   • Test essential functionalities like:
     • Login (if applicable): Send a POST request to the login endpoint with valid credentials and verify a successful response with a session token or similar mechanism.
     • Account creation (if applicable): Send a POST request to the account creation endpoint with valid data and verify a successful response with the created account details.
     • Deposit/withdrawal: Send POST requests to deposit/withdrawal endpoints with valid account numbers and amounts. Verify successful responses with updated balances or appropriate error messages for invalid requests.
     • Basic account information retrieval (if applicable): Send GET requests to endpoints that retrieve account information and verify the response includes expected details like balance.
3. Verify basic UI functionality (if applicable):
   • If your application has a basic UI (e.g., login form, deposit/withdrawal forms), manually interact with those elements to ensure proper rendering and basic functionality.

Using a Testing Framework:

1. You can leverage a testing framework like pytest to write simple smoke tests.
2. Create basic test functions that:
   • Use the client fixture provided by Flask's testing utilities to send requests to your application.
   • Assert the response status codes for essential endpoints.
   • Optionally, perform basic assertions on the response content (e.g., presence of success messages) using libraries like json to parse JSON responses.

Smoke Testing Considerations:

• Smoke tests are meant to be quick and cover the most critical functionalities.
• They shouldn't be exhaustive and don't replace more comprehensive testing approaches like unit, integration, or end-to-end tests.
• The specific functionalities you test will depend on your application's features.

Benefits of Smoke Testing:

• Early detection of critical issues: Catches major problems early in the development cycle, saving time and effort.
• Provides confidence for further testing: Helps ensure a baseline functionality level before proceeding with more in-depth testing.

Remember: Smoke testing is a starting point. As your application evolves, you'll need to adapt your smoke tests to cover new functionalities and ensure a robust testing strategy.

Ruby

greeting.rb

class Greeting
    def say_hello(name)
      "Hello, #{name}!"
    end
  end

greeting_spec.rb

require 'rspec'
require_relative 'greeting'

RSpec.describe Greeting do
  describe "#say_hello" do
    it "returns a greeting message" do
      greeting = Greeting.new
      expect(greeting.say_hello("John")).to eq("Hello, John!")
    end
  end
end

rspec greeting_spec.rb