Test & Code : Python Testing

How do you test installed packages using coverage.py? Also, a couple followups from last week's episode on using coverage for single file applications.

Have you ever written a single file Python application or script? Have you written tests for it? Do you check code coverage? This is the topic of this weeks episode, spurred on by a listener question. The questions: * For single file scripts, I'd like to have the test code included right there in the file. Can I do that with pytest? * If I can, can I use code coverage on it? The example code discussed in the episode: script.py ``` def foo(): return 5 def main(): x = foo() print(x) if name == 'main': # pragma: no cover main() test code To test: pip install pytest pytest script.py To test with coverage: put this file (script.py) in a directory by itself, say foo then from the parent directory of foo: pip install pytest-cov pytest --cov=foo foo/script.py To show missing lines pytest --cov=foo --cov-report=term-missing foo/script.py def test_foo(): assert foo() == 5 def test_main(capsys): main() captured = capsys.readouterr() assert captured.out == "5\n" ```

Building any software, including web apps and APIs requires testing. There's automated testing, and there's manual testing. In between that is exploratory testing aided by automation tools. Michael Kennedy joins the show this week to share some of the tools he uses during development and maintenance. We talk about tools used for semi-automated exploratory testing. We also talk about some of the other tools and techniques he uses to keep Talk Python Training, Talk Python, and Python Bytes all up and running smoothly. We talk about: Postman ngrok sitemap link testing scripts for manual processes using failover servers during maintenance, redeployments, etc gitHub webhooks and scripts to between fail over servers and production during deployments automatically floating IP addresses services to monitor your site: StatusCake, BetterUptime the affect of monitoring on analytics crash reporting: Rollbar, Sentry response times load testing: Locus Special Guest: Michael Kennedy.

A discussion about mocking in Python with the original contributor of unittest.mock, Michael Foord. Of course we discuss mocking and unittest.mock. We also discuss: * testing philosophy * unit testing and what a unit is * TDD * where Michael's towel is, and what color Micheal was instrumental in the building of testing tools for Python, and continues to be a pragmatic source of honest testing philosopy in a field that has a lot of contradictory information. Special Guest: Michael Foord.

Test Driven Development, TDD, is not easy to incorporate in your daily development. Martin and Brian discuss TDD and testing and Martin's experience with testing, TDD, and using it for code involved with scientific research. We discuss lots of topics around this, including: What is TDD? Should research software be tested in order to be trusted? Time pressure and the struggle to get code done quickly. How do you make time for tests also? Is testing worth it for code that will not be reused? Sometimes it's hard to know how to test something. Maybe people should learn to test alongside learning how to code. A desire for a resource of testing concepts for non-CS people. Are the testing needs and testing information needs different in different disciplines? Biology, Physics, Astrophysics, etc. Do they have different testing needs? Do we need a "how to test" resource for each? Special Guest: Martin Héroux.

Completely nerding out about pytest markers with Anthony Sottile. Some of what we talk about: Running a subset of tests with markers. Using marker expressions with and, or, not, and parentheses. Keyword expressions also can use and, or, not, and parentheses. Markers and pytest functionality that use mark, such as parametrize, skipif, etc. Accessing markers with itermarkers and get_closest_marker through item. Passing values, metadata through markers to fixtures or hook functions. Special Guest: Anthony Sottile.

MongoDB is possibly the most recognizable NoSQL document database. Mark Smith, a developer advocate for MongoDB, answers my many questions about MongoDB. We cover some basics, but also discuss some advanced features that I never knew about before this conversation. Special Guest: Mark Smith.

Visual Testing has come a long way from the early days of x,y mouse clicks and pixel comparisons. Angie Jones joins the show to discuss how modern visual testing tools work and how to incorporate visual testing into a complete testing strategy. Some of the discussion: Classes of visual testing: problems with pixel to pixel testing DOM comparisons, css, html, etc. AI driven picture level testing, where failures look into the DOM to help describe the problem. Where visual testing fits into a test strategy. Combining "does this look right" visual testing with other test workflows. "A picture is worth a thousand assertions" - functional assertions built into visual testing. Baselining pictures in the test workflow. Also discussed: - automation engineer - Test Automation University Special Guest: Angie Jones.

Scientists learn programming as they need it. Some of them learn it in college, but even if they do, that's not their focus. It's not surprising that sharing the software used for scientific research and papers is spotty, at best. And what about testing? We'd hope that the software behind scientific research is tested. But why would we expect that? We're lucky if CS students get a class or two that even mentions automated tests. Why would we expect other scientists to just know how to test their code? Martin works in research and this discussion is about software and testing in scientific research and academia. Special Guest: Martin Héroux.

Talking with Nalin Parbhu about the software evolution towards more test automation and the creation of Infuse and useMango. We talk a software development and "shift left" where automated tests and quality checks have moved earlier into the software lifecycle. Software approaches and where quality fits in Shift left Test automation Roles of software developers, SDETs (software development engineer in test), testers, QA, etc. Developers doing testing and devops Automated testing vs manual testing Regression testing, UI testing, black bock testing Unit testing, white box, API, end to end testing User acceptance testing (UAT) Mullet Methodology (Agile at the front, Waterfall at the back) Waterwheel Methodology (Requirements -> iterative development -> QA) What's an agile team? Developer resistance to testing Manifesto for agile software development Iterative development Adapting to change Agility: being able to change course quickly Special Guests: Nalin Parbhu and Ola Omiyale.

Your test suite tells you about the quality of your code under test. Mutation testing is a way to tell you about the quality your test suite. Anders Hovmöller wrote mutmut (https://mutmut.readthedocs.io/) for mutation testing in Python, and can be used with pytest, unittest, and others. In this episode, Anders explains mutation testing, how mutation testing with mutmut works, and good workflows. Special Guest: Anders Hovmöller.

Matt Harrison, author of many Python books, is putting together a course, Effective Book Authoring, to help other people write and publish books. As part of this course, he's including interviews with people who have already written books, including me. This is that interview. We discuss: * Why I wrote "Python Testing with pytest" * Self publishing vs working with a publisher * The writing, editing, and publishing process * Writing format * Book promotion * Advice to other writers Special Guest: Matt Harrison.

Wearable technology is not just smart consumer devices like watches and activity trackers. Wearable tech also includes one off projects by designers, makers, and hackers and there are more and more people producing tutorials on how to get started. Wearable tech is also a great way to get both kids and adults excited about coding, electronics, and in general, engineering skills. Sophy Wong is a designer who makes really cool stuff using code, technology, costuming, soldering, and even jewelry techniques to get tech onto the human body. Sophy joins the show to answer my many questions about getting started safely with wearable tech. Some of the questions and topics: Can I wash my clothing if I've added tech to it? Is there any danger in wearing technology or building wearable tech? Are there actual wires and cables conductive thread in the fabric and textiles of some wearable tech projects? What's a good starter project? Especially if I want to do a wearable tech project with my kids? Dealing with stretch with clothing and non-bendy electronics. Some questions around the Sophy Wong and HackSpace "Wearable Tech Projects" book. How did you get into wearable tech? Do you have a favorite project? Can I get into wearable tech if I don't know how to code or solder? Are these projects accessible to people with limited budgets? Making projects so you can reuse the expensive bits on multiple projects. Special Guest: Sophy Wong.

All test suites start fast. But as you grow your set of tests, each test adds a little bit of time to the suite. What can you do about it to keep test suites fast? Some things, like parallelization, are applicable to many domains. What about, for instance, Django applications? Well, Adam Johnson has thought about it a lot, and is here to tell us how we can speed up our Django test suites. Topics include: parallelizing tests moving from disk to memory using fake data and factory functions targeted mocking Special Guest: Adam Johnson.

Within software projects, there are lots of metrics we could measure. But which ones really matter. Instead of a list, Benjamin Harding shares with us a way of thinking about business outcomes that can help us with every day decision making. We talk about: * Business outcomes vs vanity metrics * As a developer, how do you keep business outcomes in mind * Thinking about customer value all the time * Communicating decisions and options in terms of costs and impact on business outcomes * Company culture and it's role in reinforcing a business outcome mindset * And even the role of team lead as impact multiplier I really enjoyed this conversation. But I admit that at first, I didn't realize how important this is on all software development. Metrics are front and center in a web app. But what about a service, or an embedded system with no telemetry. It still matters, maybe even more so. Little and big decisions developers face every day that have impact on costs and benefits with respect to customer value and business outcome, even if it's difficult to measure. Special Guest: Benjamin Harding.

Hackathons have been spreading around the world; many at university campuses. Major League Hacking, MLH, has been encouraging and helping hackathons. Hacking can be thought of as tinkering. Taking things apart and putting them back together as an interesting experience. There's always been some of this as part of software culture. The people at Major League Hacking have taken this to a whole new level, bringing together Tech creators who enjoy playing around with and crating new technology, on campuses, and now in virtual spaces, all over the world. Jonathon Gottfried, one of the cofounders of Major League Hacking, joins the show to talk about: hacker meetups and events hackathons what it's like to go to a hackathon how to help out with hackathons as an experienced engineer, even virtually as a mentor hackathons continuing virtually during the pandemic internships and fellowships on open source projects to help students gain experience, even during the pandemic MLH approach to internships, giving interns a support group, including peers, mentors, and project maintainers and MLH itself Special Guest: Jonathan Gottfried.

Using mock objects during testing in Python. Anna-Lena joins the podcast to teach us about mocks and using unittest.mock objects during testing. We discuss: - the different styles of using mocks - pros and cons of mocks - dependency injection - adapter pattern - mock hell - magical universe - and much more Special Guest: Anna-Lena Popkes.

Some people avoid writing tests. Some drudge through it painfully. There is a better way. In this episode, I'm going to share some advice from Luke Plant on how to "Test Smarter, Not Harder" (https://lukeplant.me.uk/blog/posts/test-smarter-not-harder/).

virtualenv supports six shells: bash, csh, fish, xonsh, cmd, posh. Each handles prompts slightly differently. Although the virtualenv custom prompt behavior should be the same across shells, Brian Skinn noticed inconsistencies. He set out to fix those inconsistencies. That was the start of an adventure in open source collaboration, shell prompt internals, difficult test problems, and continuous integration quirks. Brian Skinn initially noticed that on Windows cmd, a space was added between a prefix defined by --prompt and the rest of the prompt, whereas on bash no space was added. For reference, there were/are three nominal virtualenv prompt modification behaviors, all of which apply to the prompt changes that are made at the time of virtualenv activation: If the environment variable VIRTUAL_ENV_DISABLE_PROMPT is defined and non-empty at activation time, do not modify the prompt at all. Otherwise: If the --prompt argument was supplied at creation time, use that argument as the prefix to apply to the prompt; or, If the --prompt argument was not supplied at creation time, use the default prefix of "({{ envname }}) " as the prefix (the environment folder name surrounded by parentheses, and with a trailing space after the last paren. Special Guest: Brian Skinn.

I asked people on twitter to fill in "How do I test _____?" to find out what people want to know how to test. Lots of responses. David Lord agreed to answer them with me. In the process, we come up with lots of great general advice on how to test just about anything. Specific Questions people asked: What makes a good test? How do you test web app performance? How do you test cookie cutter templates? How do I test my test framework? How do I test permission management? How do I test SQLAlchemy models and pydantic schemas in a FastAPI app? How do I test warehouse ETL code? How do I test and mock GPIO pins on hardware for code running MicroPython on a device? How do I test PyQt apps? How do I test web scrapers? Is it the best practice to put static html in your test directory or just snippets stored in string variables? What's the best way to to test server client API contracts? How do I test a monitoring tool? We also talk about: What is the Flask testing philosophy? What do Flask tests look like? Flask and Pallets using pytest Code coverage Some of the resulting testing strategies: Set up some preconditions. Run the function. Get the result. Don't test external services. Do test external service failures. Don't test the frameworks you are using. Do test your use of a framework. Use open source projects to learn how something similar to your project tests things. Focus on your code. Focus on testing your new code. Try to architect your application such that actual GUI testing is minimal. Split up a large problem into smaller parts that are easier to test. Nail down as many parts as you can. Special Guest: David Lord.

Software tests should be order independent. That means you should be able to run them in any order or run them in isolation and get the same result. However, system state often gets in the way and order dependence can creep into a test suite. One way to fight against order dependence is to randomize test order, and with pytest, we recommend the plugin pytest-randomly to do that for you. The developer that started pytest-randomly and continues to support it is Adam Johnson, who joins us today to discuss pytest-randomly and another plugin he also wrote, called pytest-reverse. Special Guest: Adam Johnson.

Many people have been working from home now that are not used to working from home. Or at least are working from home more than they ever did before. That's definitely true for me. Even though I've been working from home since March, I wanted some tips from people who have been doing it longer. Julian Sequeira, of PyBites fame, has been working from home for about a year. Reuven Lerner, an amazing Python trainer, has been working from home for much longer. We originally had a big list of WFH topics. But we had so much fun with the tips and tricks part, that that's pretty much the whole episode. But there's lots of great tips and tricks, so I'm glad we focused on that. Special Guests: Julian Sequeira and Reuven Lerner.

Researches and others using data science and software need to follow solid software engineering practices. This is a message that Joel Grus has been promoting for some time. Joel joins the show this week to talk about data science, software engineering, and even Fizz Buzz. Topics include: Software Engineering practices and data science Difficulties with Jupyter notebooks Code reviews on experiment code Unit tests on experiment code Finding bugs before doing experiments Tests for data pipelines Tests for deep learning models Showing researchers the value of tests by showing the bugs found that wouldn't have been found without them. "Data Science from Scratch" book Showing testing during teaching Data Science "Ten Essays on Fizz Buzz" book Meditations on Python, mathematics, science, engineerign and design Testing Fizz Buzz Different algorithms and solutions to an age old interview question. If not Fizz Buzz, what makes a decent coding interview question. pytest hypothesis Math requirements for data science Special Guest: Joel Grus.

pytest 6 is out. Specifically, 6.0.1, as of July 31. And there's lots to be excited about. Anthony Sottile joins the show to discuss features, improvements, documentation updates and more. Full release notes / changelog (https://docs.pytest.org/en/stable/changelog.html) Some of what we talk about: How to update (at least, how I do it) Run your test suites with 5.4.3 or whatever the last version you were using Update to 6 Run again. Same output? Probably good. If there are any warnings, maybe fix those. You can also run with pytest -W error to turn warnings into errors. Then find out all the cool stuff you can do now New Features pytest now supports pyproject.toml files for configuration. but remember, toml syntax is different than ini files. mostly quotes are needed pytest now includes inline type annotations and exposes them to user programs. Most of the user-facing API is covered, as well as internal code. New command-line flags --no-header and --no-summary A warning is now shown when an unknown key is read from a config INI file. The --strict-config flag has been added to treat these warnings as errors. New required_plugins configuration option allows the user to specify a list of plugins, including version information, that are required for pytest to run. An error is raised if any required plugins are not found when running pytest. Improvements You can now pass output to things like less and head that close the pipe passed to them. thank you!!! Improved precision of test durations measurement. use --durations=10 -vv to capture and show durations Rich comparison for dataclasses and attrs-classes is now recursive. pytest --version now displays just the pytest version, while pytest --version --version displays more verbose information including plugins. --junitxml now includes the exception cause in the message XML attribute for failures during setup and teardown. Improved Documentation Add a note about --strict and --strict-markers and the preference for the latter one. Explain indirect parametrization and markers for fixtures. Bug Fixes Deprecations Trivial/Internal Changes Breaking Changes you might need to care about before upgrading PytestDeprecationWarning are now errors by default. Check the deprecations and removals (https://docs.pytest.org/en/latest/deprecations.html) page if you are curious. -k and -m internals were rewritten to stop using eval(), this results in a few slight changes but overall makes them much more consistent testdir.run().parseoutcomes() now always returns the parsed nouns in plural form. I'd say that's an improvement Special Guest: Anthony Sottile.

pip is the package installer for Python. Often, when you run pip, especially the first time in a new virtual environment, you will see something like: WARNING: You are using pip version 20.1.1; however, version 20.2 is available. You should consider upgrading via the 'python -m pip install --upgrade pip' command. And you should. Because 20.2 has a new dependency resolver. Get in the habit, until October, of replacing pip install with pip install --use-feature=2020-resolver. This flag is new in the 20.2 release. This new pip dependency resolver is the result of a lot of work. Five of the people involved with this work are joining the show today: Bernard Tyers, Nicole Harris, Paul Moore, Pradyun Gedam, and Tzu-ping Chung. We talk about: * pip dependency resolver changes * user experience research and testing * crafting good error messages * efforts to improve the test suite * testing pip with pytest * some of the difficulties with testing pip * working with a team on a large project * working with a large code base * bringing new developers into a large project Special Guests: Bernard Tyers, Nicole Harris, Paul Moore, Pradyun Gedam, and Tzu-ping Chung.

Lots of Python projects are starting to use GitHub Actions for Continous Integration & Deployment (CI/CD), as well as other workflows. Tania Allard, a Senior Cloud Developer Advocate at Microsoft, joins the show to answer some of my questions regarding setting up a Python project to use Actions. Some of the topics covered: How to get started with GitHub Actions for a Python project? What are workflow files? Does it matter what the file name is called? Can I have / Should I have more than one workflow? Special Guest: Tania Allard.

A great resume is key to landing a great software job. There's no surprise there. But so many people make mistakes on their resume that can very easily be fixed. Randall Kanna is on the show today to help us understand how to improve our resumes, and in turn, help us have better careers. Special Guest: Randall Kanna.

Len Wanger works on industrial 3D printers. And I was pleased to find out that there's a bunch of Python in those printers as well. In this episode we talk about: 3D printers What are the different types of 3D printers? Where are 3D printed industrial parts being used? Why use one type of additive manufacturing over another? Python in 3D printing hardware. What are Finite State Machines, FSMs? Benefits of FSMs for testing, logging, and breaking a complex behavior into small testable parts. Benefits of simulation in writing and testing software to control hardware. Special Guest: Len Wanger.

FastAPI is a modern, fast (high-performance), web framework for building APIs with Python based on standard Python type hints. Typer is a library for building CLI applications, also based on Python type hints. Type hints and many other details are intended to make it easier to develop, test, and debug applications using FastAPI and Typer. The person behind FastAPI and Typer is Sebastián Ramírez. Sebastián is on the show today, and we discuss: FastAPI Rest APIs Swagger UI Future features of FastAPI Starlette Typer Click Testing with Typer and Click Typer autocompletion Typer CLI Special Guest: Sebastián Ramírez.

There's stuff going on in Python packaging and pyproject.toml. Brett and I talk about some upcoming work on Python packaging, such as: editable installs the need for standardization is a feature of setuptools that's so tools like flit can create stub setup.py files for editable installs configuration of other tools in pyproject.toml originally not supposed to happen, was supposed to be only a build tool black and other tools use it anyway now it's ok And then get off on tangents and talk about: why it's good to have packages like pip, toml, setuptools, wheel, etc not part of the standard library should we remove some stuff from the standard library the standard library using unittest for testing the standard library why not hypothesis I didn't bring up "why not pytest?" but you know I was thinking it. why CPython and not C++Python and more Special Guest: Brett Cannon.

Code Coverage or Test Coverage is a way to measure what lines of code and branches in your code that are utilized during testing. Coverage tools are an important part of software engineering. But there's also lots of different opinions about using it. - Should you try for 100% coverage? - What code can and should you exclude? - What about targets? I've been asked many times what I think about code coverage or test coverage. This episode is a train of thought brain dump on what I think about code coverage. We'll talk about: - how I use code coverage to help me write source code - line coverage and branch coverage - behavior coverage - using tests to ask and answer questions about the system under test - how to target coverage just to the code you care about - excluding code - good reasons and bad reasons to exclude code And also the Pareto Principle or 80/20 rule, and the law of diminishing returns and how that applies (or doesn't) to test coverage.

The Python extension for VS Code is most downloaded extension for VS Code. Brett Cannon is the manager for the distributed development team of the Python extension for VS Code. In this episode, Brett and I discuss the Python extension and VS Code, including: pytest support virtual environment support how settings work, including user and workspace settings multi root projects testing Python in VS Code debugging and pydevd jump to cursor feature upcoming features Special Guest: Brett Cannon.

pytest plugins are an amazing way to supercharge your test suites, leveraging great solutions from people solving test problems all over the world. In this episode Michael and I discuss 15 favorite plugins that you should know about. We also discuss fixtures and plugins and other testing tools that work great with pytest * tox * GitHub Actions * Coverage.py * Selenium + splinter with pytest-splinter * Hypothesis And then our list of pytest plugins: 1. pytest-sugar 1. pytest-cov 1. pytest-stress 1. pytest-repeat 1. pytest-instafail 1. pytest-metadata 1. pytest-randomly 1. pytest-xdist 1. pytest-flake8 1. pytest-timeout 1. pytest-spec 1. pytest-picked 1. pytest-freezegun 1. pytest-check 1. fluentcheck That last one isn't a plugin, but we also talked about pytest-splinter at the beginning. So I think it still counts as 15. Special Guest: Michael Kennedy.

One of the great things about attending in person coding conferences, such as PyCon, is the hallway track, where you can catch up with people you haven't seen for possibly a year, or maybe even the first time you've met in person. Nina is starting something like the hallway track, online, on twitch, and it's already going, so check out the first episode of Python Tea (https://www.twitch.tv/videos/635831555). Interesting coincidence is that this episode is kind of like a hallway track discussion between Nina and Brian. We've had Nina on the show a couple times before, but it's been a while. In 2018, we talked about Mentoring on episode 44 (https://testandcode.com/44). In 2019, we talked about giving Memorable Tech Talks in episode 71 (https://testandcode.com/71). In this episode, we catch up with Nina, find out what she's doing, and talk about a bunch of stuff, including: Live Coding Online Conferences Microsoft Python team Python Tea, an online hallway track Q&A with Python for VS Code team Python on hardware Adafruit Device Simulator Express CircuitPython Tricking out your command prompt Zsh and Oh My Zsh Emacs vs vi key bindings for shells Working from home Special Guest: Nina Zakharenko.

"The mission of the Python Software Foundation is to promote, protect, and advance the Python programming language, and to support and facilitate the growth of a diverse and international community of Python programmers." That's a lot of responsibility, and to that end, the PSF Board Directors help out quite a bit. If you want to be a part of the board, you can. There's an election coming up right around the corner and you gotta get your nomination in by May 31. You can also join the PSF if you want to vote for who gets to be part of the board. But what does it really mean to be on the Board, and what are some of the things the PSF does? To help answer those questions, I've got Ewa Jodlowska, the PSF Executive Director, and Christopher Neugebauer, a current board member, on the show today. I've also got some great links in the show notes if we don't answer your questions and you want to find out more. Special Guests: Christopher Neugebauer and Ewa Jodlowska.

Technical debt has to be dealt with on a regular basis to have a healthy product and development team. The impacts of technical debt include emotional drain on engineers and slowing down development and can adversely affect your hiring ability and retention. But really, what is technical debt? Can we measure it? How do we reduce it, and when? James Smith, the CEO of Bugsnag, joins the show to talk about technical debt and all of these questions. Special Guest: James Smith.

"Code is read much more often than it is written." - Guido van Rossum This is true for both production code and test code. When you are trying to understand why a test is failing, you'll be very grateful to the test author if they've taken the care to make it readable. David Seddon came up with 6 principles to help us write more readable tests. We discuss these, as well as more benefits of readable tests. David's 6 Principles of Readable Tests: Profit from the work of others Put naming to work Show only what matters Don’t repeat yourself Arrange, act, assert Aim high Special Guest: David Seddon.

In both unittest and pytest, when a test function hits a failing assert, the test stops and is marked as a failed test. What if you want to keep going, and check more things? There are a few ways. One of them is subtests. Python's unittest introduced subtests in Python 3.4. pytest introduced support for subtests with changes in pytest 4.4 and a plugin, called pytest-subtests. Subtests are still not really used that much. But really, what are they? When could you use them? And more importantly, what should you watch out for if you decide to use them? That's what Paul Ganssle and I will be talking about today. Special Guest: Paul Ganssle.

Django supports testing out of the box with some cool extensions to unittest. However, many people are using pytest for their Django testing, mostly using the pytest-django plugin. Adam Parkin, who is known online as CodependentCodr (https://twitter.com/codependentcodr), joins us to talk about migrating an existing Django project from unittest to pytest. Adam tells us just how easy this is. Special Guest: Adam Parkin.

Financial services have their own unique testing development challenges. But they also have lots of the same challenges as many other software projects. Eric Bergemann joins Brian Okken to discuss: Specific testing challenges in the financial services domain CI/CD : Continuous Integration, Continuous Deployment TDD : Test Driven Development Confidence from testable applications Testing strategies to add coverage to legacy systems Testing the data and test cases themselves DevOps Continuous testing Manual testing procedures BDD & Gherkin Hiring in vs training industry knowledge Special Guest: Eric Bergemann.

Apache Spark is a unified analytics engine for large-scale data processing. PySpark blends the powerful Spark big data processing engine with the Python programming language to provide a data analysis platform that can scale up for nearly any task. Johnathan Rioux, author of "PySpark in Action", joins the show and gives us a great introduction of Spark and PySpark to help us decide how to get started and decide whether or not to decide if Spark and PySpark are right you. Special Guest: Jonathan Rioux.

Hypothesis is the Python tool used for property based testing. Hypothesis claims to combine "human understanding of your problem domain with machine intelligence to improve the quality of your testing process while spending less time writing tests." In this episode Alexander Hultnér introduces us to property based testing in Python with Hypothesis. Some topics covered: What is property based testing Thinking differently for property based testing Using hypothesis / property based testing in conjunction with normal testing Failures saved and re-run What parts of development/testing is best suited for hypothesis / property based testing Comparing function implementations Testing against REST APIs that use Open API / Swagger with schemathesis Changing the number of tests in different test environments System, integration, end to end, and unit tests Special Guest: Alexander Hultnér.

IDEs can help people with automated testing. In this episode, Paul Everitt and Brian discuss ways IDEs can encourage testing and make it easier for everyone, including beginners. We discuss features that exist and are great, as well as what is missing. The conversation also includes topics around being welcoming to new contributors for both open source and professional projects. We talk about a lot of topics, and it's a lot of fun. But it's also important. Because IDEs can make testing Some topics discussed: Making testing more accessible Test First vs teaching testing last TDD workflow Autorun Rerunning last failures Different ways to run different levels of tests Command line flags and how to access them in IDEs pytest.ini zooming in and out of test levels running parametrizations running tests with coverage and profiling parametrize vs parameterize parametrization identifiers pytest fixture support global configurations / configuration templates coverage and testing and being inviting to new contributors confidence in changes and confidence in contributions navigating code, tests, fixtures grouping tests in modules, classes, directories BDD, behavior driven development, cucumber, pytest-bdd web development testing parallel testing with xdist and IDE support refactor rename Special Guest: Paul Everitt.

The Test Anything Protocol, or TAP, is a way to record test results in a language agnostic way, predates XML by about 10 years, and is still alive and kicking. Matt Layman has contributed to Python in many ways, including his educational newsletter, and his Django podcast, Django Riffs. Matt is also the maintainer of tap.py and pytest-tap, two tools that bring the Test Anything Protocol to Python. In this episode, Matt and I discuss TAP, it's history, his involvement, and some cool use cases for it. Special Guest: Matt Layman.

pytest is awesome by itself. pytest + plugins is even better. In this episode, Anthony Sottile and Brian Okken discuss the top 28 pytest plugins. Some of the plugins discussed (we also mention a few plugins related to some on this list): pytest-cov pytest-timeout pytest-xdist pytest-mock pytest-runner pytest-instafail pytest-django pytest-html pytest-metadata pytest-asyncio pytest-split-tests pytest-sugar pytest-rerunfailures pytest-env pytest-cache pytest-flask pytest-benchmark pytest-ordering pytest-watch pytest-pythonpath pytest-flake8 pytest-pep8 pytest-repeat pytest-pylint pytest-randomly pytest-selenium pytest-mypy pytest-freezegun Honorable mention: - pytest-black - pytest-emoji - pytest-poo Special Guest: Anthony Sottile.

Django is without a doubt one of the most used web frameworks for Python. Lacey Williams Henschel is a Django consultant and has joined me to talk about Django, the Django community, and so much more. Topics: * Django * The Django Community * Django Girls * Django Girls Tutorial * DjangoCon * Software Testing * Using tests during learning * pytest-django * testing Django * Wagtail Special Guest: Lacey Williams Henschel.

Harry Percival has completed his second book, "Architecture Patterns with Python". So of course we talk about the book, also known as "Cosmic Python". We also discuss lots of testing topics, especially related to larger systems and systems involving third party interfaces and APIs. Topics Harry's new book, "Architecture Patterns with Python". a.k.a. Cosmic Python TDD : Test Driven Development Test Pyramid Tradeoffs of different architectural choices Mocks and their pitfalls Avoiding mocks Separating conceptual business logic Dependency injection Dependency inversion Identifying external dependencies Interface adapters to mimize the exposed surface area of external dependencies London School vs Classic/Detroit School of TDD Testing strategies for testing external REST APIs Special Guest: Harry Percival.

Application security is best designed into a system from the start. Anthony Shaw is doing something about it by creating an editor plugin that actually helps you write more secure application code while you are coding. On today's Test & Code, Anthony and I discuss his security plugin, but also application security in general, as well as other security components you need to consider. Security is something every team needs to think about, whether you are a single person team, a small startup, or a large corporation. Anthony and I also discuss where to start if it's just a few of you, or even just one of you. Topics include: Finding security risks while writing code. What are the risks for your applications. Thinking about attack surfaces. Static and dynamic code analysis. Securing the environment an app is running in. Tools for scanning live sites for vulnerabilities. Secret management. Hashing algorithms. Authentication systems. and Anthony's upcoming cPython Internals book. Special Guest: Anthony Shaw.

Let's say you have a web application and you want to make some changes to improve it. You may want to A/B test it first to make sure you are really improving things. But really what is A/B testing? That's what we'll find out on this episode with Leemay Nassery. Special Guest: Leemay Nassery.

I play a form of group chess that has some interesting analogies to software development and maintenance of existing systems. This episode explains group chess and explores a few of those analogies.

pytest-testmon is a pytest plugin which selects and executes only tests you need to run. It does this by collecting dependencies between tests and all executed code (internally using Coverage.py) and comparing the dependencies against changes. testmon updates its database on each test execution, so it works independently of version control. In this episode, I talk with testmon creator Tibor Arpas about testmon, about it's use and how it works. Special Guest: Tibor Arpas.

This episode is not just a look back on 2019, and a look forward to 2020. Also, 2019 is the end of an amazingly transofrmative decade for me, so I'm going to discuss that as well. top 10 episodes of 2019 10: episode 46 (https://testandcode.com/46), Testing Hard To Test Applications - Anthony Shaw 9: episode 64 (https://testandcode.com/64), Practicing Programming to increase your value 8: episode 70 (https://testandcode.com/70), Learning Software without a CS degree - Dane Hillard 7: episode 75 (https://testandcode.com/75), Modern Testing Principles - Alan Page 6: episode 72 (https://testandcode.com/72), Technical Interview Fixes - April Wensel 5: episode 69 (https://testandcode.com/69), Andy Hunt - The Pragmatic Programmer 4: episode 73 (https://testandcode.com/73), PyCon 2019 Live Recording 3: episode 71 (https://testandcode.com/71), Memorable Tech Talks, The Ultimate Guide - Nina Zakharenko 2: episode 76 (https://testandcode.com/76), TDD: Don’t be afraid of Test-Driven Development - Chris May 1: episode 89 (https://testandcode.com/89), Improving Programming Education - Nicholas Tollervey Looking back on the last decade Some amazing events, like 2 podcasts, a book, a blog, speaking events, and teaching has led me to where we're at now. Looking forward to 2020 and beyond I discussed what's in store in the next year and moving forward. A closing quote Software is a blast. At least, it should be. I want everyone to have fun writing software. Leaning on automated tests is the best way I know to allow me confidence and freedome to: - rewrite big chunks of code - play with the code - try new things - have fun without fear - go home feeling good about what I did - be proud of my code I want everyone to have that. That's why I promote and teach automated testing. I hope you had an amazing decade. And I wish you a productive and fun 2020 and the upcoming decade. If we work together and help eachother reach new heights, we can achieve some pretty amazing things

Pipelines are used a lot in software projects to automated much of the work around build, test, deployment and more. Thomas Eckert talks with me about pipelines, specifically Azure Pipelines. Some of the history, and how we can use pipelines for modern Python projects. Special Guest: Thomas Eckert.

Data science and machine learning are affecting more of our lives every day. Decisions based on data science and machine learning are heavily dependent on the quality of the data, and the quality of the data pipeline. Some of the software in the pipeline can be tested to some extent with traditional testing tools, like pytest. But what about the data? The data entering the pipeline, and at various stages along the pipeline, should be validated. That's where pipeline tests come in. Pipeline tests are applied to data. Pipeline tests help you guard against upstream data changes and monitor data quality. Abe Gong and Superconductive are building an open source project called Great Expectations. It's a tool to help you build pipeline tests. This is quite an interesting idea, and I hope it gains traction and takes off. Special Guest: Abe Gong.

You've applied for a job, maybe lots of jobs. Depending on the company, you've gotta get through: a resume review a coding challange a phone screen maybe another code example an in person interview If you get the job, and you enjoy the work, awesome, congratulations. If you don't get the job, it'd be really great to know why. Sometimes it isn't because you aren't a skilled engineer. What other reasons are there? Well, that's what we're talking about today. Charity Majors is the cofounder and CTO of Honeycomb.io, and we're going to talk about reasons for not hiring someone. This is a very informative episode both for people who job hunt in the future and for hiring managers and people on the interview team. Special Guest: Charity Majors.

Andy Knight is the Automation Panda. Andy Knight is passionate about software testing, and shares his passion through public speaking, writing on automationpanda.com, teaching as an adjunct professor, and now also through writing a book and organizing a new regional Python conference. Topics of this episode include: Andy's book on software testing Being an adjunct professor Public speaking and preparing talk proposals including tips from Andy about proposals and preparing for talks PyCarolinas Special Guest: Andy Knight.

Cristian Medina wrote an article recently called "Test Engineering Anti-Patterns: Destroy Your Customer Satisfaction and Crater Your Quality By Using These 9 Easy Organizational Practices" Of course, it's sarcastic, and aims to highlight many problems with organizational practices that reduce software quality. The article doesn't go out of character, and only promotes the anti-patterns. However, in this interview, we discuss each point, and the corollary of what you really should do. At least, our perspectives. Here's the list of all the points discussed in the article and in this episode: Make the Test teams solely responsible for quality Require all tests to be automated before releasing Require 100% code coverage Isolate the Test organization from Development Measure the success of the process, not the product. Metrics, if rewarded, will always be gamed. Require granular projections from engineers Reward quick patching instead of solving Plan for today instead of tomorrow Special Guest: Cristian Medina.

Python 3.8.0 final is live and ready to download. On todays episode, we're going to run through what's new, picking out the bits that I think are the most interesting and affect the most people, including new language features standard library changes optimizations in 3.8 Not just the big stuff everyone's already talking about. But also some little things that will make programming Python even more fun and easy. I'm excited about Python 3.8. And really, this episode is to my way to try to get you excited about it too.

pytest 5.2 was just released, and with it, a cool fun feature called dynamic scope fixtures. Anthony Sottile so tilly is one of the pytest core developers, so I thought it be fun to have Anthony describe this new feature for us. We also talk about parametrized testing and really what is fixture scope and then what is dynamic scope. Special Guest: Anthony Sottile.

Nicholas Tollervey is working toward better ways of teaching programming. His projects include the Mu Editor, PyperCard, and CodeGrades. Many of us talk about problems with software education. Nicholas is doing something about it. Special Guest: Nicholas Tollervey.

Tools like error monitoring, crash reporting, and performance monitoring are tools to help you create a better user experience and are fast becoming crucial tools for web development and site reliability. But really what are they? And when do you need them? You've built a cool web app or service, and you want to make sure your customers have a great experience. You know I advocate for utilizing automated tests so you find bugs before your customers do. However, fast development lifecycles, and quickly reacting to customer needs is a good thing, and we all know that complete testing is not possible. That's why I firmly believe that site monitoring tools like logging, crash reporting, performance monitoring, etc are awesome for maintaining and improving user experience. John-Daniel Trask, JD, the CEO of Raygun, agreed to come on the show and let me ask all my questions about this whole field. Special Guest: John-Daniel Trask.

There's a cool feature of pytest called parametrization. It's totally one of the superpowers of pytest. It's actually a handful of features, and there are a few ways to approach it. Parametrization is the ability to take one test, and send lots of different input datasets into the code under test, and maybe even have different output checks, all within the same test that you developed in the simple test case. Super powerful, but something since there's a few approaches to it, a tad tricky to get the hang of.

You've incorporated software testing into your coding practices and know from experience that it helps you get your stuff done faster with less headache. Awesome. Now your colleagues want in on that super power and want to learn testing. How do you help them? That's where Josh Peak is. He's helping his team add testing to their workflow to boost their productivity. That's what we're talking about today on Test & Code. Josh walks us through 4 maxims of developing software tests that help grow your confidence and proficiency at test writing. Special Guest: Josh Peak.

Good software testing strategy is one of the best ways to save developer time and shorten software development delivery cycle time. Software test suites grow from small quick suites at the beginning of a project to larger suites as we add tests, and the time to run the suites grows with it. Fortunately, pytest has many tricks up it's sleave to help shorten those test suite times. Niklas Meinzer is a software developer that recentely wrote an article on optimizing test suites. In this episode, I talk with Niklas about the optimization techniques discussed in the article and how they can apply to just about any project. Special Guest: Niklas Meinzer.

Adafruit enables beginners to make amazing hardware/software projects. With CircuitPython, these projects can now use Python. The combination of Python's ease of use and Adafruit's super cool hardware and a focus on a successful beginner experience makes learning to write code that controls hardware super fun. In this episode, Scott Shawcroft, the project lead, talks about the past, present, and future of CircuitPython, and discusses the focus on the beginner. We also discuss contributing to the project, testing CircuitPython, and many of the cool projects and hardware boards that can use CircuitPython, and Blinka, a library to allow you to use "CircuitPython APIs for non-CircuitPython versions of Python such as CPython on Linux and MicroPython," including Raspberry Pi. Special Guest: Scott Shawcroft.

Bob Belderbos and Julian Sequeira started PyBites (https://pybit.es/) a few years ago. They started doing code challanges along with people around the world and writing about it. Then came the codechalleng.es (https://codechalleng.es/) platform, where you can do code challenges in the browser and have your answer checked by pytest tests. But how does it all work? Bob joins me today to go behind the scenes and share the tech stack running the PyBites Code Challenges platform. We talk about the technology, the testing, and how it went from a cool idea to a working platform. Special Guest: Bob Belderbos.

Anthony Sottile is a pytest core contributor, as well as a maintainer and contributor to many other projects. In this episode, Anthony shares some of the super cool features of pytest that have been added since he started using it. We also discuss Anthony's move from user to contributor, and how others can help with the pytest project. Special Guest: Anthony Sottile.

In the last episode, we talked about going from script to supported package. I worked on a project called subark and did the packaging with flit. Today's episode is a continuation where we add new features to a supported package and how to develop and test a flit based package. Covered: viewing stages of a project with git tags flit support for editable installs flit description entry in pyproject.toml to put README on pypi. development dependencies in pyproject.toml editor layout for optimal TDD-ing test case grouping modifications to traditional TDD that helps me develop faster. code and command snippets from episode: For git checkout of versions: $ git clone https://github.com/okken/submark.git $ cd submark $ python3 -m venv venv --prompt submark $ source ./bin/activate (submark) $ git checkout v0.1 ... etc ... (submark) $ git checkout v0.7 To grab the latest again: (submark) $ git checkout master pyproject.toml change for README to show up on pypi: [tool.flit.metadata] ... description-file = "README.md" ... Adding dev dependencies to pyproject.toml: [tool.flit.metadata.requires-extra] test = ["pytest", "pytest-cov", "tox"] Installing in editable mode (in top level repo directory). works in mac, linux, windows: (submark) $ flit install --pth-file or for mac/linux: (submark) $ flit install -s

This episode is a story about packaging, and flit, tox, pytest, and coverage. And an alternate solution to "using the src". Python makes it easy to build simple tools for all kinds of tasks. And it's great to be able to share small projects with others on your team, in your company, or with the world. When you want to take a script from "just a script" to maintainable package, there are a few steps, but none of it's hard. Also, the structure of the code layout changes to help with the growth and support. Instead of just talking about this from memory, I thought it'd be fun to create a new project and walk through the steps, and report back in a kind of time lapse episode. It should be fun. Here are the steps we walk through: 0.1 Initial script and tests 0.2 build wheel with flit 0.3 build and test with tox 0.4 move source module into a package directory 0.5 move tests into tests directory

Some information about software testing is just wrong. I'm not talking about opinions. I have lots of opinions and they differ from other peoples opinions. I'm talking about misinformation and old information that is no longer applicable. I've ran across a few lateley that I want to address. All of the following are wrong: Integrated tests can't work. I can prove it with wacky math. Tests have to be blazing fast or they won't get run. TDD is about design, not about testing. This episode discusses why these are wrong.

Roadblocks to writing tests, and what to do about it. Some developers either don't write tests, or don't like writing tests. Why not? I love writing tests. In this episode we examine lots of roadblocks to testing, and start coming up with solutions for these.

Creating maintainable test suites for complex systems. The episode describes some complexities involved with hardware testing, then shares techniques for shifting complexity out of the test cases. quick overview of what test instruments are discussion of API and communication with instruments techniques for shifting complexity out of test cases These techniques should apply to all test suites dealing with complex systems: Creating test cases that are easy to read and debug and tell a story about what is being tested. Pushing setup complexity into fixtures. Pushing lengthy repetitive API call sets into helper functions. Using stable, documented, interfaces.

Test Driven Development, TDD, can be intimidating to try. Why is that? And how can we make it less scary? That's what this episode is about. Chris May is a Python developer and the co-founder of PyRVA, the Richmond Virginia Python group. In this episode, Chris shares his experience with adding testing and TDD to his work flow. I really enjoyed talking with Chris, and I think his story will help lots of people overcome testing anxiety. Special Guest: Chris May.

Software testing, if done right, is done all the time, throughout the whole life of a software project. This is different than the verification and validation of a classical model of QA teams. It's more of a collaborative model that actually tries to help get great software out the door faster and iterate quicker. One of the people at the forefront of this push is Alan Page. Alan and his podcast cohost Brent Jensen tried to boil down what modern testing looks like in the Modern Testing Principles. I've got Alan here today, to talk about the principles, and also to talk about this transition from classical QA to testing specialists being embedded in software teams and then to software teams doing their own testing. But that only barely scratches the surface of what we cover. I think you'll learn a lot from this discussion. The seven principles of Modern Testing (http://moderntesting.org): Our priority is improving the business. We accelerate the team, and use models like Lean Thinking and the Theory of Constraints to help identify, prioritize and mitigate bottlenecks from the system. We are a force for continuous improvement, helping the team adapt and optimize in order to succeed, rather than providing a safety net to catch failures. We care deeply about the quality culture of our team, and we coach, lead, and nurture the team towards a more mature quality culture. We believe that the customer is the only one capable to judge and evaluate the quality of our product We use data extensively to deeply understand customer usage and then close the gaps between product hypotheses and business impact. We expand testing abilities and knowhow across the team; understanding that this may reduce (or eliminate) the need for a dedicated testing specialist. Special Guest: Alan Page.

In this episode, I talk with Derrick Mar, CTO and co-founder of Pathrise. This is the episode you need to listen to to get ready for software interviews. We discuss four aspects of technical interviews that interviewers are looking for: communication problem solving coding verification How to practice for the interview. Techniques for synchronizing with interviewer and asking for hints. Even how to ask the recruiter or hiring manager how to prepare for the interview. If you or anyone you know has a software interview coming up, this episode will help you both feel more comfortable about the interview before you show up, and give you concrete tips on how to do better during the interview. Special Guest: Derrick Mar.

This is a "Yay! It's PyCon 2019" episode. PyCon is very important to me. But it's kinda hard to put a finger on why. So I figured I'd ask more people to help explain why it's important. I ask a few simple questions to people about Python and PyCon and get some great insights into both the language popularity and the special place this conference holds to many people.

Some typical technical interview practices can be harmful and get in the way of hiring great people. April Wensel offers advice to help fix the technical interview process. She recommends: * hire for mindset and attitude * look for empathy and mentorship skills * allow candidates to show their strengths instead of hunting for weaknesses * have the candidate leave feeling good about themselves and your company, regardless of the hiring decision Some topics discussed: * interview questions to bring out stories of skills and successes * stereotype threat * diversity * interview hazing * white boards * coding challenges * unconscious bias * emotional intelligence * myth of talent shortage * pair programming and collaboration during interviews * mirrortocracy * cultural add vs cultural fit * empathy * mentoring This episode is important for anyone going into a technical interview, as a candidate, as a hiring manager, or as a member of an interview team. Special Guest: April Wensel.

Nina Zakharenko gives some great advice about giving tech talks. We talk about a blog series that Nina wrote called "The Ultimate Guide To Memorable Tech Talks". This episode is full of great help and encouragement for your own public speaking adventures. Some of what we discuss: * overcoming the fear of public speaking * breathing and pausing during talks * planning your talk as well as planning your time to get ready for the talk * writing proposals and getting feedback on proposals * Nina's talk in PyCascades on programming Adafruit chips * types of talks that are often rejected * pre-recording demos to avoid live demo problems * why you should speak, even if you are an introvert * benefits of public speaking * a super cool announcement at the end Special Guest: Nina Zakharenko.

Dane and Brian discuss skills needed for people that become software developers from non-traditional paths. Dane is also writing a book to address many of these skill gaps, Code Like a Pro (https://www.manning.com/books/code-like-a-pro), that's currently in an early access phase. Use code podtest&code19 to get a discount. And, sign up as a Friend of the Show (https://testandcode.com/friends-of-the-show) to enter for a chance to win a free copy of the eBook version. We also discuss the writing process, testing with a multi-language stack, music, art, photography, and more. Special Guest: Dane Hillard.

Andy Hunt and Dave Thomas wrote the seminal software development book, The Pragmatic Programmer. Together they founded The Pragmatic Programmers and are well known as founders of the agile movement and authors of the Agile Manifesto. They founded the Pragmatic Bookshelf publishing business in 2003. The Pragmatic Bookshelf published it's most important book, in my opinion, in 2017 with the first pytest book (https://pragprog.com/book/bopytest/python-testing-with-pytest) available from any publisher. Topics: * The Pragmatic Programmer (https://pragprog.com/book/tpp/the-pragmatic-programmer), the book * The Manifesto for Agile Software Development (https://agilemanifesto.org/) * Agile methodologies and lightweight methods * Some issues with "Agile" as it is now. * The GROWS Method (https://growsmethod.com/) * Pragmatic Bookshelf (https://pragprog.com/), the publishing company * How Pragmatic Bookshelf is different, and what it's like to be an author (http://write-for-us.pragprog.com/) with them. * Reading and writing sci-fi novels, including Conglommora (https://conglommora.com/), Andy's novels. * Playing music (https://andyhunt.bandcamp.com/). Special Guest: Andy Hunt.

With conventional TDD, you write a failing test, get it to pass, then refactor. Then run the tests again to make sure your refactoring didn't break anything. But what if it did break something? Kent Beck has been recommending to commit your code to revision control after every green test run. Oddmund Strømme suggested a symmetrical idea to go ahead and revert the code when a test fails. Kent writes that he hated the idea, but had to try it. Then wrote about it last September. And now we have TCR, "(test && commit) || revert". What's it feel like to actually do this? Well, Thomas Deniffel has been using it since about a month after that article came out. In this episode, we'll hear from Thomas about his experience with it. It's a fascinating idea. Have a listen and let me know what you think. Special Guest: Thomas Deniffel.

In today's episode we talk with Kelly Paredes & Sean Tibor. They teach Python in a middle school in Florida, and talk about this experience on the podcast "Teaching Python". I love that they include physical computing right from the start, and everything else they are doing. It's a fun interview. Special Guests: Kelly Paredes and Sean Tibor.

I was recently interviewed on a podcast called "IT Career Energizer Podcast". Phil Burgess is the host of the podcast, and it was a lot of fun. I think it turned out well, and I wanted to share it with you here, with Phil's permission, of course. Special Guest: Phil Burgess.

Is it ok to have more than one assert statement in a test? I've seen articles that say no, you should never have more than one assert. I've also seen some test code made almost unreadable due to trying to avoid more than one assert per test. Where did this recommendation even come from? What are the reasons? What are the downsides to both perspectives? That's what we're going to talk about today.

I want you to get the most out of being a software developer, or test engineer, or whatever you do that makes this podcast relevant to your life. By "get the most" I mean: the most fun the most value more career options probably more responsibility maybe even more money, that'd be cool I want you to start (or continue) studying and practicing your skills. But not just random practice, I've got a strategy to help you focus what to study. Why am I talking about this now? Here's some background on how I re-learned how to have fun with code refactoring (https://testandcode.com/pybites) through code challenges. I'm going to write up the whole list as a blog post, which I'll share first with my Patreon Supporters (https://www.patreon.com/testpodcast), second with my email list (https://us5.list-manage.com/subscribe?u=1f123c581ab0df2737f3174b9&id=9db722df54) and slack channel (http://pythontesting.net/slack/) and then as an actual post somewhere.

I hear and I forget. I see and I remember. I do and I understand. -- Confucius Matt Harrison is an author and instructor of Python and Data Science. This episode focuses on his training company, MetaSnake, and corporate training. Matt's written several books on Python, mostly self published. So of course we talk about that. But the bulk of the conversation is about corporate training, with Brian playing the role of someone considering starting a corporate training role, and asking Matt, an experienced expert in training, how to start and where to go from there. I think you'll learn a lot from this. Special Guest: Matt Harrison.

There are a lot of learning styles and a lot of ways to learn Python. If you started Python through a class at work, or through an online course, or maybe an email series, it's possibly you may have learned from Reuven Lerner. If your first encounter with pytest was reading an article in Linux Journal recently, that would be the writing of Reuven. Reuven Lerner teaches Python. This interview definitely falls into the category of talking with interesting people doing interesting things with Python. We talk about how incorporating testing into teaching can add a level of clarity to the interaction and help people duirng the learning process. I'm also fascinated by people who teach and train because it's a skill I'm trying to improve. Special Guest: Reuven Lerner.

A look back on 3 years of podcasting, and a bit of a look forward to what to expect in 2019. Top 5 episodes: 2: Pytest vs Unittest vs Nose (https://testandcode.com/2) 33: Katharine Jarmul - Testing in Data Science (https://testandcode.com/33) 18: Testing in Startups and Hiring Software Engineers with Joe Stump (https://testandcode.com/18) 45: David Heinemeier Hansson - Software Development and Testing, TDD, and exploratory QA (https://testandcode.com45) 27: Mahmoud Hashemi : unit, integration, and system testing (https://testandcode.com/27) Honorable mention: 32: David Hussman - Agile vs Agility, Dude's Law, and more (https://testandcode.com/32) This episode also went through lots of: what went well what was lacking what's next Please listen and let me know where I should take this podcast.

Julian Sequeira is Co-Founder of PyBit.es (a blog/platform created to teach and learn Python) and a Python Trainer at Talk Python Training. He's also a survivor of the 100DaysOfCode in Python Challenge. We talk about the 100 days challenge, about learning Python, and about how cool it is to learn within a community. Special Guest: Julian Sequeira.

Thea Flowers is a Pythonista and open source advocate. She helps empower developers of all backgrounds and experience levels using Python and open source software and hardware. Thea is the creator of Nox, the co-chair of PyCascades 2019, the lead maintainer of urllib3, and a member of the Python Packaging Authority and Packaging Working Group. Thea works on Google Cloud Platform's wonderful Developer Relations team where she works on API client libraries and community outreach. All of that is definitely cool enough. But she is also building a synthesiser based on Sega Genesis chips. So of course, that's where we'll start the conversation. Special Guest: Thea Flowers.

Let's say you've got a web application you need to test. It has a REST API that you want to use for testing. Can you use Python for this testing even if the application is written in some other language? Of course. Can you use pytest? duh. yes. what else? What if you want to spin up docker instances, get your app running in that, and run your tests against that environment? How would you use pytest to do that? Well, there, I'm not exactly sure. But I know someone who does. Dima Spivak is the Director of Engineering at StreamSets, and he and his team are doing just that. He's also got some great advice on utilizing code reviews across teams for test code, and a whole lot more. Special Guest: Dima Spivak.

Data science, data engineering, data analysis, and machine learning are part of the recent massive growth of Python. But really what is data science? Vicki Boykis helps me understand questions like: No really, what is data science? What does a data pipeline look like? What is it like to do data science, data analysis, data engineering? Can you do analysis on a laptop? How big does data have to be to be considered big? What are the challenges in data science? Does it make sense for software engineers to learn data engineering, data science, pipelines, etc? How could someone start learning data science? Also covered: A type work (analysis) vs B type work (building) data lakes and data swamps predictive models data cleaning development vs experimentation Jupyter Notebooks Kaggle ETL pipelines I learned a lot about the broad field of data science from talking with Vicki. Special Guest: Vicki Boykis.

Michael Kennedy of Talk Python and Python Bytes fame joins Brian to talk about being a great guest and what to expect. Even if you have never wanted to be on a podcast, you might learn some great tips. A few of the things we talk about will be helpful for other endeavors, like public speaking, guest blog posts, look for unsolicited job opportunities. Some people have never been on a podcast before, and are possibly freaked out about some of the unknowns of being on a podcast. That's why we did this episode. Michael and I discuss a bunch of the niggly details so that you can be relaxed and know what to expect. Topics include: If you want to be on a podcast How to stand out and be someone a podcast would want to have on a show. How to suggest yourself as a guest and the topic you want to discuss. Picking a topic for a podcast What to do before the show to prepare Helping the host out with some information Some hardware (not much) Some software (all free) Sending info like bio, headshot, links, etc. What to expect the host or show to do before the recording. Where to record Sketching out some show topics with the host, maybe on a shared document. What to expect and do Right before the show During the conversation After the recording When it goes live (help promote it) Special Guest: Michael Kennedy.

What happens when 100% test code coverage just isn't enough. In this episode, we talk with Mahmoud Hashemi about glom, a very cool project in itself, but a project that needs more coverage than 100%. This problem affects lots of projects that use higher level programming constructs, like domain specific languages (DSLs), sub languages mini languages, compilers, and db query languages. Also covered: * awesome Python applications * versioning: 0-ver vs calver vs semver Special Guest: Mahmoud Hashemi.

Paul talks about the beginning years of Python. Talking about Python's beginnings is also talking about the Python community beginnings. Yes, it's reminiscing, but it's fun. Special Guest: Paul Everitt.

Luc Perkins joins the show to talk about "Seven Databases in Seven Weeks: A guide to modern databases and the NoSQL movement." We discuss a bit about each database: Redis, Neo4J, CouchDB, MongoDB, HBase, Postgres, and DynamoDB. Special Guest: Luc Perkins.

Brett Cannon discusses the changes afoot in Python packaging as a result of PEP 517, PEP 518, starting with "How did we get here?" and "Where are we going?" Discussed: flit Poetry tox Continuous Integration setup.py, MANIFEST.in, etc. pipenv what's with lock files applications (doesn't go on PyPI) vs libraries (goes on PyPI) workflows dependency resolution deployment dependencies vs development dependencies will lock files be standarized multiple lock files requirements.txt Special Guest: Brett Cannon.

Andy Knight joins me in discussing the concept of feature testing. A feature tests is "a test verifying a service or library as the customer would use it, but within a single process." That was a quote from an article that appeared on the Twitter engineering blog. The article describes a shift away from class tests towards feature tests, the benefits of the shift, and some reactions to it. Feature tests are similar to something I used to call "functional subcutaneous integration test", but it's a way better name, and I plan to use it more often. The idea fits well with my testing philosophy. Andy Knight is someone still holding onto the testing pyramid. So I thought it would be fun to ask him to discuss feature testing with me. I think it's a balanced discussion. I hope you enjoy it and learn something. Special Guest: Andy Knight.

Anthony Shaw joins Brian to discuss flaky tests and flaky test suites. What are flaky tests? Is it the same as fragile tests? Why are they bad? How do we deal with them? What causes flakiness? How can we fix them? How can we avoid them? Proactively rooting out flakiness Test design GUI tests Sharing solutions Special Guest: Anthony Shaw.

tox is a simple yet powerful tool that is used by many Python projects. tox is not just a tool to help you test a Python project against multiple versions of Python. In this interview, Oliver and Brian just scratch the surface of this simple yet powerful automation tool. This is from the tox documentation: tox is a generic virtualenv management and test command line tool you can use for: checking your package installs correctly with different Python versions and interpreters running your tests in each of the environments, configuring your test tool of choice acting as a frontend to Continuous Integration servers, greatly reducing boilerplate and merging CI and shell-based testing. Yet tox is so much more. It can help create development environments, hold all of your admin scripts, ... I hope you enjoy this wonderful discussion of tox with Oliver Bestwalter, one of the core maintainers of tox. Special Guest: Oliver Bestwalter.

The story of how I came to find a good user interface for running and debugging automated tests is interleaved with a multi-year effort of mine to have a test workflow that’s works smoothly with product development and actually speeds things up. It’s also interleaved with the origins of the blog pythontesting.net, this podcast, and the pytest book I wrote with Pragmatic. It’s not a long story. And it has a happy ending. Well. It’s not over. But I’m happy with where we are now. I’m also hoping that this tale of my dedication to, or obsession with, quality and developer efficiency helps you in your own efforts to make your daily workflow better and to extend that to try to increase the efficiency of those you work with.

Interview with Andy Knight, the Automation Panda. * Selenium & WebDriver * Headless Chrome * Gherkin * BDD * Given When Then * pytest-bdd * PyCharm * Writing Good Gherkin * Overhead of Gherkin and if it's worth it * When to use pytest vs pytest-bdd * The art of test automation Special Guest: Andy Knight.

How do you write tests for things that aren’t that easy to write tests for? That question is a possibly terrible summary of a question sent to me by a listener. And to help me start answering that question, I asked a friend of mine to help, Antony Shaw. Of course, different types of applications have different test strategies, so there’s not a universal answer. But I know some of you out there have experience and expertise around how to tackle this problem. Listen to the discussion Anthony and I have about it, and let me know if you have some techniques or tips to add. Special Guest: Anthony Shaw.

David Heinemeier Hansson is the creator of Ruby on Rails, founder & CTO at Basecamp (formerly 37signals). He's a best selling author, public speaker, and even a Le Mans class winning racing driver. All of that, of course, is awesome. But that's not why I asked him on the show. In 2014, during a RailsConf keynote, he started a discussion about damage caused by TDD. This was followed by a few blog posts, and then a series of recorded hangouts with Martin Fowler and Kent Beck. This is what I wanted to talk with David about; this unconventional yet practical and intuitive view of how testing and development work together. It's a great discussion. I think you'll get a lot out of it. Special Guest: David Heinemeier Hansson.

Nina Zakharenko is a cloud developer advocate at Microsoft focusing on Python. She's also an excellent public speaker. We talk about her experience with mentoring, both being a mentor, and utilizing mentors. We also talk about public speaking, her move to Microsoft, and to Portland, and the Microsoft/GitHub merge. Special Guest: Nina Zakharenko.

After I had wrapped up the interview with Kelsey Hightower for episode 43 (http://testandcode.com/43), I asked him one last question. You see, I admire the his presentation style. So I asked him if he would share with me how he prepared for his presentations. His answer is so thoughtful and makes so much sense, I couldn't keep it to myself. I'm releasing this as a bonus mini-episode so that it's easy to refer back to the next time you or I have a chance to do a technical talk. Special Guest: Kelsey Hightower.

I first heard Kelsey speak during his 2017 PyCon keynote. He's an amazing speaker, and I knew right then I wanted to hear more about what he does and hear more of his story. We discuss testing, of course, but we take it further and discuss: tests for large systems, like kubernetes Testing in real world scenarios with all the configuration and everything Becoming a complete engineer by thinking about the end to end flow from the users perspective Learning from other roles, and the value of roles to allow focus and expertise We even get into Chaos Engineering and testing live systems. Special Guest: Kelsey Hightower.

This interview with Trey Hunner discusses his use of automated tests to help teach programming. Automated testing is a huge part of developing great software. But many new developers don't get exposed to testing for quite a while. But this is changing. New ways to teach programming include automated tests from the beginning. Trey Hunner is one of the PSF directors and a Python and Django team trainer, and he has been using automated tests to help people learn Python. Special Guest: Trey Hunner.

We talk with Anthony Shaw about some of the testing problems facing both DevOps teams, and Agile teams. We also talk about his recent pull request accepted into pytest. Special Guest: Anthony Shaw.

Adam is the host of The Gently Mad (https://thegentlymad.com/) podcast, and teaches the steps in creating and growing a podcast in his course Irresistible Podcasting (https://irresistiblepodcasting.com). He was one of the people who inspired Brian to get the Test & Code podcast started in the first place. Brian took his course in 2015. Adam is in the process of updating the course, and building a community around it. Warning: This may be an episode to listen to with headphones if you have kids around. There is swearing. I wanted to get Adam's help to convince many of you to either come on this show as a guest, or start your own podcast. We did some of that. But we also cover a lot of issues like self doubt and the importance of community. Special Guest: Adam Clark.

Complete and exhaustive testing is not possible. Nor would it be fun, or maintainable, or a good use of your time. However, some functionality is important enough to make sure the test behavior coverage is thorough enough to have high confidence in it's quality. In this episode, we discuss 3 techniques that can be combined to quickly generate test cases. We then talk about how to implement them efficiently in pytest. The techniques covered are: equivalence partitioning boundary value analysis decision tables We discuss how to use these to generate test cases for a new list filter functionality in the cards application. The resulting tests: 1 UI test to make sure the options are able to be passed in correctly. 1 small parametrized test function with 16 single line parameter sets representing the different test cases.

RCRCRC was developed by Karen Nicole Johnson. In this episode we discuss the mnemonic/heuristic and use it to prioritize tests for the cards application. Recent: new features, new areas of code Core: essential functions must continue to work, your products USPs (Unique Selling Propositions) Risk: some areas of an application pose more risk, perhaps areas important to customers but not used regularly by the development team. Configuration sensitive: code that’s dependent on environment settings or operating system specifics Repaired: tests to reproduce bugs, tests for functionality that has been recently repaired. Chronic: functionality that frequently breaks

This episode starts down the path of test strategy with the first tests to write in either a legacy system or a project just getting off it's feet. We cover: My approach to testing existing systems. Put names to strategies so we can refer to them later. Explain the strategies in general terms and explain why they are useful. Discuss how these strategies are used in an example project. (The code is available on github). Strategies covered today: Dog Fooding Exploratory Testing Tracer Bullet Tests Act Like A Customer (ALAC) Tests Manual Procedures Initial automated tests at 2 levels, API and UI.

Stephanie is a co-founder and graphics engineer at Binomial. She works on Basis, an image compressor, and has customers in games, video, mapping, and any application that has lots of image data. Stephanie has also been encouraging experienced engineers to open up their twitter DMs to questions from anyone, to help mentor people not only in technical questions, but in career questions as well. She also sets aside some time to mentor people through skype when written form just doesn't cut it. That's the primary reason I have Stephanie on today, to talk about mentoring and open office hours. But we also talk about - Binomial - image compression - texture mapping - the use of both manual and automated testing for complex systems - sane work hours - work life balance - and how long hours have led her to the opinions she holds today Special Guest: Stephanie Hurlburt.

There are lots of ways to up your skills. Of course, I'm a big fan of learning through reading books, such as upping your testing skills by reading Python Testing with pytest. And then there are online learning systems and MOOCs. At the other end of the spectrum is a full blown university degree. One option kind of in the middle is continuing education programs available through some universities, such as University of Washington. To discuss this option with me in more depth, we've got Andrew Hoover, Senior Director, Program Strategy, University of Washington Continuum College Special Guest: Andrew Hoover.

An in depth discussion of Test Driven Development (TDD) should include a discussion of Test First. So that's where we start. Why write tests first? How do you know what tests to write? What are the steps for test first? Isn't this just TDD? Functional Tests vs Unit Tests

A discussion with Katharine Jarmul, aka kjam, about some of the challenges of data science with respect to testing. Some of the topics we discuss: experimentation vs testing testing pipelines and pipeline changes automating data validation property based testing schema validation and detecting schema changes using unit test techniques to test data pipeline stages testing nodes and transitions in DAGs testing expected and unexpected data missing data and non-signals corrupting a dataset with noise fuzz testing for both data pipelines and web APIs datafuzz hypothesis testing internal interfaces documenting and sharing domain expertise to build good reasonableness intermediary data and stages neural networks speaking at conferences Special Guest: Katharine Jarmul.

A wonderful discussion with David Hussman. David and Brian look back at what all we've learned in XP, TDD, and other Agile methodologies, where things have gone awry, how to bring the value back, and where testing fits into all of this. How to build the wrong thing faster Agile vs Agility Product vs Process Where testing fits into software development practices. "Integration tests, there's a name that needs to be refactored desperately." Integration tests are "story tests". They tell the story of the product. XP and TDD and the relationship with tests To test for design, use microtests, xUnit style. User Advocy tests are often lacking, but are needed to learn about the product. "I just keep writing tests until I'm not scared anymore." - Kent Beck Dude's Law: Value = Why/How People often focus so much on the how that they forget about why they are doing something. Subcutaneous Tests "The hardest part of programming is thinking." Refactoring vs Repaving Agility means being able to quickly change direction During experimentation and learning, what matters isn't how much you got done, but how much you learn. "The best way to get automation is to make developers do manual tests." Special Guest: David Hussman.

What started as a twitter disagreement carries over into this civil discussion of software testing. Brian and Paul discuss testing practices such as the testing pyramid, TDD, unit testing, system testing, and balancing test effort. the Testing Pyramid the Testing Column TDD unit testing balancing unit with system tests, functional tests API testing subcutaneous testing customer facing tests Special Guest: Paul Merrill.

M. Scott Ford is the founder and chief code whisperer at Corgibytes, a company focused on helping other companies with legacy code. Topics include: How M. Scott Ford got into forming a company that works on legacy code. Technical debt Process debt Software testing The testing pyramid iterative development kanban readable code and readable test code Special Guest: M. Scott Ford.

Kobiton is a service to test mobile apps on real devices. QASymphony offers software testing and QA tools. Special Guest: Josh Lieberman.

Today we have an interview with Casey Rosenthal of Netflix. One of the people making sure Netflix runs smoothly is Casey Rosenthall. He is the manager for the Traffic, Intuition, and Chaos teams at Netflix. He's got a great perspective on quality and large systems. We talk about Chaos Engineering Experimentation vs Testing Testing Strategy Visualization of large amounts of data representing Steady State Special Guest: Casey Rosenthal.

What is the difference between a unit test, an integration test, and a system test? Mahmoud Hashemi helps me to define these terms, as well as discuss the role of all testing variants in software development. What is the difference between a unit test, an integration test, and a system test? TDD testing pyramid vs testing column the role of testing in software development web frameworks listen to wikipedia hatnote the world’s largest photo competition Enterprise Software with Python Links: Mahmoud on twitter: @mhashemi (https://twitter.com/mhashemi) Mahmoud on sedimental (http://sedimental.org/) hatnote (http://blog.hatnote.com/) listen to wikipedia (http://listen.hatnote.com/) Montage (https://blog.wikimedia.org/2016/12/22/montage-platform-wiki-loves-monuments/), the web platform used to help judge the world’s largest photo competition clastic (https://pypi.python.org/pypi/clastic) 10 Myths of Enterprise Python (http://sedimental.org/10_myths_of_enterprise_python.html) Enterprise Software with Python (http://shop.oreilly.com/product/0636920047346.do?code=authd) course Enterprise Software with Python (http://sedimental.org/esp.html) blog post. Special Guest: Mahmoud Hashemi.

Interview with Sam Van Oort about pyresttest (https://github.com/svanoort/pyresttest), "A REST testing and API microbenchmarking tool" pyresttest A question in the Test & Code Slack channel (http://pythontesting.net/slack) was raised about testing REST APIs. There were answers such as pytest + requests, of course, but there was also a mention of pyresttest, https://github.com/svanoort/pyresttest (https://github.com/svanoort/pyresttest), which I hadn't heard of. I checked out the github repo, and was struck by how user friendly the user facing test definitions were. So I contacted the developer, Sam Van Oort, and asked him to come on the show and tell me about this tool and why he developed it. Here's the "What is it?" section from the pyresttest README: A REST testing and API microbenchmarking tool Tests are defined in basic YAML or JSON config files, no code needed Minimal dependencies (pycurl, pyyaml, optionally future), making it easy to deploy on-server for smoketests/healthchecks Supports generate/extract/validate mechanisms to create full test scenarios Returns exit codes on failure, to slot into automated configuration management/orchestration tools (also supplies parseable logs) Logic is written and extensible in Python Support Special thanks to my wonderful Patreon supporters (http://patreon.com/testpodcast) and those who have supported the show by purchasing Python Testing with unittest, nose, pytest (http://pythontesting.net/book)

Interview with Dave Hunt @davehunt82 (https://twitter.com/davehunt82). We Cover: Selenium Driver (http://www.seleniumhq.org/) pytest (http://docs.pytest.org/) pytest plugins: pytest-selenium (http://pytest-selenium.readthedocs.io/) pytest-html (https://pypi.python.org/pypi/pytest-html) pytest-variables (https://pypi.python.org/pypi/pytest-variables) tox (https://tox.readthedocs.io) Dave Hunt’s “help wanted” list on github (https://github.com/search?utf8=%E2%9C%93&q=author%3Adavehunt+type%3Aissue+label%3A%22help+wanted%22+state%3Aopen+no%3Aassignee) Mozilla (https://www.mozilla.org) Also: fixtures xfail CI and xfail and html reports CI and capturing pytest code sprint working remotely for Mozilla

pytest is an extremely popular test framework used by many projects and companies. In this episode, I interview Raphael Pierzina (@hackebrot (https://twitter.com/hackebrot)), a core contributor to both pytest and cookiecutter. We discuss how Raphael got involved with both projects, his involvement in cookiecutter, pytest, "adopt pytest month", the pytest code sprint, and of course some of the cool new features in pytest 3. Links: Raphael Pierzina on twitter (@hackebrot (https://twitter.com/hackebrot)) pytest - http://doc.pytest.org (http://doc.pytest.org/en/latest/) cookie cutter - https://github.com/audreyr/cookiecutter (https://github.com/audreyr/cookiecutter) cookiecutter-pytest-plugin - https://github.com/pytest-dev/cookiecutter-pytest-plugin (https://github.com/pytest-dev/cookiecutter-pytest-plugin)

Kent Beck's twitter profile says "Programmer, author, father, husband, goat farmer". But I know him best from his work on extreme programming, test first programming, and test driven development. He's the one. The reason you know about TDD is because of Kent Beck. I first ran across writings from Kent Beck as started exploring Extreme Programming in the early 2000's. Although I don't agree with all of the views he's expressed in his long and verbose career, I respect him as one of the best sources of information about software development, engineering practices, and software testing. Along with Test First Programming and Test Driven Development, Kent started an automated test framework that turned into jUnit. jUnit and it's model of setup and teardown wrapping test functions, as well base test class driven test frameworks became what we know of as xUnit style frameworks now, which includes Python's unittest. He discussed this history and a lot more on episode 122 of Software Engineering Radio. The episode is titled "The History of JUnit and the Future of Testing with Kent Beck", and is from Sept 26, 2010. http://www.se-radio.net/2010/09/episode-167-the-history-of-junit-and-the-future-of-testing-with-kent-beck/ I urge you to download it and listen to the whole thing. It's a great interview, still relevant, and applicable to testing in any language, including Python. What I've done in this podcast is take a handful of clips from the interview (with permission from IEEE and SERadio), and discuss the clips and my opinions a bit. The lessons are: You're tests should tell a story. Be careful of DRY, inheritance, and other software development practices that might get in the way of keeping your tests easy to understand. All test should help differentiate good programs from bad programs and not be redundant. Test at multiple levels and multiple scales where it makes sense. Differentiating between TDD, BDD, ATDD, etc. isn't as important as testing your software to learn about it. Who cares what you call it.

How do you convert manual tests to automated tests? This episode looks at the differences between manual and automated tests and presents two strategies for converting manual to automated.

A listener requested that I start covering some terminology. I think it's a great idea. Covered in this episode: Test Fixtures Subcutaneous Testing End to End Testing (System Testing) I also discuss: A book rewrite Progress on transcripts A story from the slack channel

I talk with Michael about: Episodes of his show having to do with testing. His transition from employee to podcast host and online training entrepreneur. His Python training courses. The Pyramid Web framework. Courses by Michael Explore Python Jumpstart by Building 10 Apps Explore Write Pythonic Code Like a Seasoned Developer Python for Entrepreneurs Testing related podcast Episodes from Talk Python To Me: episode 10: Harry Percival, TDD for the Web in Python, and PythonAnywhere PythonAnywhere Harry's book, TDD with Python episode 45: Brian Okken, Pragmatic testing and the Testing Column Talk Python To Me podcast episode 63: Austin Bingham, Mutation Testing, Cosmic Ray Cosmic Ray episode 67: David MacIver, Hypothesis Hypothesis

Interview with Robert Collins, current core maintainer of Python's unittest module. Some of the topics covered How did Robert become the maintainer of unittest? unittest2 as a rolling backport of unittest test and class parametrization with subtest and testscenarios Which extension to unittest most closely resembles Pytest fixtures? Comparing Pytest and unittest Will unittest ever get assert rewriting? Future changes to unittest I've been re-studying unittest recently and I mostly wanted to ask Robert a bunch of clarifying questions. This is an intermediate to advanced discussion of unittest. Many great features of unittest go by quickly in this talk. Please let me know if there's something you'd like me to cover in more depth as a blog post or a future episode. Links unittest (https://docs.python.org/3.5/library/unittest.html) unittest2 (https://pypi.python.org/pypi/unittest2) pip (https://docs.python.org/3.5/installing/) mock (https://docs.python.org/dev/library/unittest.mock.html) testtools (https://testtools.readthedocs.io/en/latest/) fixtures (https://pypi.python.org/pypi/fixtures) testscenarios (https://pypi.python.org/pypi/testscenarios) subunit (https://pypi.python.org/pypi/python-subunit) pipserver (https://pypi.python.org/pypi/pypiserver) devpi (https://pypi.python.org/pypi/devpi-server) testresources (https://pypi.python.org/pypi/testresources) TIP (testing in python) mailing list (http://lists.idyll.org/listinfo/testing-in-python)

In this episode, I interview with Joe Stump, cofounder of Sprintly (https://sprint.ly), to give the startup perspective to development and testing. Joe has spent his career in startups. He's also been involved with hiring and talent acquisition for several startups. We talk about testing, continuous integration, code reviews, deployment, tolerance to defects, and how some of those differ between large companies and small companies and startups. Then we get into hiring. Specifically, finding and evaluating good engineers, and then getting them to be interested in working for you. If you ever want to grow your team size, you need to listen to this.

The Travis Foundation. Interview with Laura Gaetano Links and things we talked about: Travis Foundation (http://foundation.travis-ci.org) Open Source Grants (http://foundation.travis-ci.org/grants/) The Foundation's support of Katrina Owen from exercism.io (http://foundation.travis-ci.org/2016/01/25/exercism/) Exercism.io (http://Exercism.io) Rails Girls summer of code (http://railsgirlssummerofcode.org/campaign/) Diversity Tickets (http://diversitytickets.org) Conference support Speakerinnen (http://speakerinnen.org) Prompt (http://mhprompt.org/)

This is a small episode. I'm changing the name from the "Python Test Podcast" to "Test & Code". I just want to discuss the reasons behind this change, and take a peek at what's coming up in the future for this podcast. Links The Waterfall Model and "Managing the Development of Large Software Systems" (http://testandcode.com/7) Josh Kalderimis from Travis CI (http://tesatandcode.com/14)

An introduction to Lean Software Development This is a quick intro to the concepts of Lean Software Development. I'm starting a journey of trying to figure out how to apply lean principles to software development in the context of 2016/2017. Links Lean Software Development (http://amzn.to/223fkLo) book by Mary & Tom Poppendieck wikipedia (https://en.wikipedia.org/wiki/Lean_software_development) entry for Lean Software Development Patreon supporters of the show (http://patreon.com/testpodcast) Talk Python to Me (https://talkpython.fm/) Podcast Python Jumpstart by Building 10 Apps - video course (https://www.kickstarter.com/projects/mikeckennedy/python-jumpstart-by-building-10-apps-video-course?ref=card) pytest sprint (http://pytest.org/latest/announce/sprint2016.html) pytest.org (http://pytest.org) pytest/tox indiegogo campaign (https://www.indiegogo.com/projects/python-testing-sprint-mid-2016#/)

Interview with Josh Kalderimis from Travis CI. Josh is a co-founder and Chief Post-It Officer at Travis CI. Topics What is Continuous Integration, CI What is Travis CI Some history of the company travis-ci.org vs travis-ci.com and merging the two Enterprise and the importance of security Feature questions Travis vs Jenkins Travis notification through Slack Reporting history of Travis results Dealing with pytest results status other than pass/fail Capturing std out and stderr logging from tests Build artifacts Tox and Travis Using Selenium What does a Chief Post-It Officer do Differentiation between Travis and other CI options Using Slack to keep remote teams communicating well Travis team Funding open source projects Travis Foundation Rails Girls Summer of Code Open source grants Mustaches and beards Shite shirts New Zealand What does Team Periwinkle do Links Jeff Knupp's Open Sourcing a Python Project the Right Way (https://www.jeffknupp.com/blog/2013/08/16/open-sourcing-a-python-project-the-right-way/) Sven's blog post when Travis started (http://svenfuchs.com/2011/2/5/travis-a-distributed-build-server-tool-for-the-ruby-community) Sven's mustache and Josh's beard (https://travis-ci.com/about) Travis CI for open source (http://travis-ci.org) Travis CI for private repositories and enterprise (http://travis-ci.com) Slack (https://slack.com/) Travis Foundation (http://foundation.travis-ci.org/) Rails Girls Summer of Code (http://railsgirlssummerofcode.org/) Talk Python to Me Podcast (https://talkpython.fm/episodes/show/45/the-python-testing-column-now-a-thing)

Testing apps that use requests without using mock. Interview with Ian Cordasco (@sigmavirus24 (https://github.com/sigmavirus24)) Topics: Betamax - python library for replaying requests interactions for use in testing. requests github3.py Pycon 2015 talk: Ian Cordasco - Cutting Off the Internet: Testing Applications that Use Requests - PyCon 2015 Pytest and using Betamax with pytest fixtures The utility (or uselessness) of teaching programming with Java (My own rant mainly) Rackspace and Ian’s role at Rackspace and OpenStack Python Code Quality Authority: flake8, pep8, mccabe, pylint, astroid, … Static code analysis and what to use which tool when. Raymond Hettinger - Beyond PEP 8 -- Best practices for beautiful intelligible code - PyCon 2015 Links: Testing Python-Requests with Betamax (https://semaphoreci.com/community/tutorials/testing-python-requests-with-betamax) Cutting Off the Internet: Testing Applications that Use Requests - PyCon 2015 (https://youtu.be/YHbKxFcDltM?t=1m55s) github3.py (https://pypi.python.org/pypi/github3.py) requests (http://docs.python-requests.org/en/master/) Rackspace (https://www.rackspace.com/) Openstack (https://www.openstack.org/) Python Code Quality Authority (https://github.com/PyCQA) and documentation (http://meta.pycqa.org/en/latest/) GitLab (https://about.gitlab.com/) Raymond Hettinger - Beyond PEP 8 -- Best practices for beautiful intelligible code - PyCon 2015 (https://www.youtube.com/watch?v=wf-BqAjZb8M) Other Betamax resources: Betamaxing Boto3 (http://www.roadsi.de/betamaxing-boto3.html) Using Betamax with pytest fixtures (http://www.coglib.com/~icordasc/blog/2015/07/betamax-050-now-with-a-pytest-fixture.html) Isolated @memoize (http://nedbatchelder.com/blog/201601/isolated_memoize.html)

In this episode I interview Ned Batchelder. I know that coverage.py is very important to a lot of people to understand how much of their code is being covered by their test suites. Since I'm far from an expert on coverage, I asked Ned to discuss it on the show. I'm also quite a fan of Ned's 2014 PyCon talk "Getting Started Testing", so I definitely asked him about that. We also discuss edX, Python user groups, PyCon talks, and more. Some of what's covered (pun intended) in this episode: coverage.py types of coverage Line coverage branch coverage Behavior coverage Data coverage How Ned became the owner of coverage.py Running tests from coverage.py vs running coverage from test runner. edX what is it what Ned's role is Ned's blog Ned's PyCon 2014 talk "Getting Started Testing" Teaching testing and the difficulty of the classes being part of unittest fixtures package some of the difficulties of teaching unittest because of it's class based system. the history of classes in unittest coming from java's jUnit implementation Boston's Python Group PyCon in Portland Ned to do a talk here "Machete mode debugging". Practicing PyCon talks at local group meetings. At the very least, practice it in front of a live audience. Links: Ned Batchelder (http://nedbatchelder.com/) Coverage (https://pypi.python.org/pypi/coverage) Coverage documentation (https://coverage.readthedocs.org) django-nose (https://pypi.python.org/pypi/django-nose) pytest-django (https://pypi.python.org/pypi/pytest-django) edX (https://www.edx.org/) open edX (https://open.edx.org/) Boston Python User Group (http://www.meetup.com/bostonpython/) Portland Python User Group (http://www.meetup.com/pdxpython/) - I need to go to these PyCon 2016 (https://us.pycon.org/2016/) - Planning on attending, it's in Portland. Yay! Getting Started Testing (http://nedbatchelder.com/text/test0.html) - Ned's 2014 Pycon talk

How pytest, unittest, and nose deal with assertions. The job of the test framework to tell developers how and why their tests failed is a difficult job.
In this episode I talk about assert helper functions and the 3 methods pytest uses to get around having users need to use assert helper functions.

Given-When-Then is borrowed from BDD and is my favorite structure for test case design. It doesn’t matter if you are using pytest, unittest, nose, or something completely different, this episode will help you write better tests. The Given-When-Then structure for test method/function development. How and why to utilize fixtures for your given or precondition code. Similarities with other structure discriptions. Setup-Test-Teardown Setup-Excercise-Verify-Teardown. Arrange-Act-Assert Preconditions-Trigger-Postconditions. Benefits Communicate the purpose of your test more clearly Focus your thinking while writing the test Make test writing faster Make it easier to re-use parts of your test Highlight the assumptions you are making about the test preconditions Highlight what outcomes you are expecting and testing against. Links discussed in the show: Mechanics of pytest, unittest, nose (http://pythontesting.net/start-here) unittest fixture reference (http://pythontesting.net/framework/unittest/unittest-fixtures/) nose fixture reference (http://pythontesting.net/framework/nose/nose-fixture-reference/) pytest fixtures (series of posts starting here) (http://pythontesting.net/framework/pytest/pytest-fixtures/) pytest style fixtures (http://pythontesting.net/framework/pytest/pytest-fixtures-nuts-bolts) pytest parameterized fixtures (http://pythontesting.net/framework/pytest/pytest-fixtures-nuts-bolts/#params)

Intro to Harry Percival, his background and story of how he got into TDD and ended up writing a book (http://amzn.to/1SqW1t3) Comparing using unittest and pytest with applicability to testing django projects. Functional end to end testing with selenium. The django test client for middle level tests. test isolation django and isolated unit tests unit tests vs integration tests Testing done by the development team without an external QA Double loop TDD: Functional test first, then unit tests Spikes: investigations without tests Harry's experience with having a freely available web version of a book that is also intended to be sold. Update: Comment from Harry Percival on 19-Jan-2014 I might have been a bit down on unit tests vs functional tests in that "unit tests never fail comment". Not true at all, particularly as we've just been thru upgrading django on our core system, and the unit tests really saved our bacon on that one... Links Test-Driven Development with Python (http://amzn.to/1SqW1t3) Obey the Testing Goat (http://pythontesting.net/obey-the-testing-goat) - Harry's site dedicated to the book and related posts. Python Testing with unittest, nose, pytest (http://pythontesting.net/book) Gary Bernhardt's talk, Boundaries talk (http://pythontesting.net/boundaries-talk) including a discussion of "Functional Core, Imperative Shell". Video of Boundaries talk on youtube (http://pythontesting.net/boundaries-talk-youtube) Special Guest: Harry Percival.

In today's podcast, I dodge the question of "What do you think of Agile?" by reading an essay from Dave Thomas (http://pragdave.me/blog/2014/03/04/time-to-kill-agile/)

The waterfall model has been used and modified and changed and rebelled against since before I started programming. Waterfall such an important character in the story of software development that we should get to know it a better.

My experience with writing software comes from my experience: where I grew up, what eras I lived through, what my economical and geographical experiences have been, when I learned to code, and what projects I've worked on.

Setup and Teardown Benefits of Test Fixtures code reuse cleanup of resources errors vs failures focusing your thinking on what you are testing and what you are not scoping for efficiency Brief look at pytest named fixtures References pytest fixtures series (http://pythontesting.net/framework/pytest/pytest-fixtures/) pytest fixtures nuts & bolts (http://pythontesting.net/framework/pytest/pytest-fixtures-nuts-bolts/) pytest session scoped fixtures (http://pythontesting.net/framework/pytest/pytest-session-scoped-fixtures/) unittest fixtures (http://pythontesting.net/framework/unittest/unittest-fixtures/) nose fixtures mentioned in introduction (http://pythontesting.net/framework/nose/nose-introduction/#fixtures) nose fixture reference post (http://pythontesting.net/framework/nose/nose-fixture-reference/) how to run a single class (http://pythontesting.net/framework/specify-test-unittest-nosetests-pytest/)

Setup and Teardown Benefits of Test Fixtures code reuse cleanup of resources errors vs failures focusing your thinking on what you are testing and what you are not scoping for efficiency Brief look at pytest named fixtures References pytest fixtures series (http://pythontesting.net/framework/pytest/pytest-fixtures/) pytest fixtures nuts & bolts (http://pythontesting.net/framework/pytest/pytest-fixtures-nuts-bolts/) pytest session scoped fixtures (http://pythontesting.net/framework/pytest/pytest-session-scoped-fixtures/) unittest fixtures (http://pythontesting.net/framework/unittest/unittest-fixtures/) nose fixtures mentioned in introduction (http://pythontesting.net/framework/nose/nose-introduction/#fixtures) nose fixture reference post (http://pythontesting.net/framework/nose/nose-fixture-reference/)

Answering a listener question. Why testing? What are the benefits? Why automated testing over manual testing? Why test first? Why do automated testing during development? Why test to the user level API? After describing my ideal test strategy and project, I list: Business related, practical benefits of testing Personal reasons to embrace testing Pragmatic, day to day, developer benefits of testing

I list my requirements for a framework and discuss how Pytest, Unittest, and Nose measure up to those requirements. Mentioned: pytest (http://pythontesting.net/framework/pytest/pytest-introduction/) unittest (http://pythontesting.net/framework/unittest/unittest-introduction/) nose (http://pythontesting.net/framework/nose/nose-introduction/) delayed assert (http://pythontesting.net/strategy/delayed-assert/) pytest-expect (http://pythontesting.net/pytest-expect/) doctest (http://pythontesting.net/framework/doctest/doctest-introduction/) I did the audio processing differently for this episode. Please let me know how it sounds, if there are any problems, etc.