Quantcast
Channel: Testpad Blog
Viewing all 42 articles
Browse latest View live

Welcome to Testpad

$
0
0

Welcome to Testpad and, more specifically, Testpad's blog. The plan is to use this blog to keep you up to date with what's happening with the app, its features and, for the geeks at heart, how it works. But first, a bit of context...

What is Testpad?

Testpad is a web app (i.e. you point your browser at http://ontestpad.com) that gives you a simple and fast way to write your test scripts. It's not to be confused with tools that help automate testing; Testpad is for the testing you have to do by hand. You can think of it as an online document editor, highly specialised to the task of writing lists of things to test, recording results and reporting on progress.

Testpad is great for testing mobile apps, websites, and software in general. In fact, Testpad would be great in any testing/checklist scenario, not just software. But that said, it has been built with the problems and language of software testing in mind, so will seem most relevant to the challenge of shipping good software.

Simple by design, and blazingly fast

It's simple to write tests, simple to run tests and simple to report on tests. Testpad's goal is to be one BIG step up from sharing around copies of spreadsheets, yet way, way cheaper than the kind of money you can spend on fully-fledged Test Case Management.

Testpad is fast. Not just fast in the sense that Google is fast at page loading, but fast in actual use. With Testpad it's very fast and convenient to write a script and run it, over and over again.

Testpad makes a lot of use of asynchronous javascript to talk to the servers. This means you can focus on typing your test cases, whilst Testpad does all the saving and result tracking in the background. Plus, there are lots of keyboard shortcuts so you don't even have to reach for the mouse; just keep typing.

Foundations

Testpad is built on great technology including several of Amazon's cloud services (EC2, ELB, EBS, SES, Route53), and some awesome opensource projects including MongoDB, Django, and jQuery... but that's the subject of future blog posts - stay tuned if you're interested. Meanwhile, why not...

Give it a go!

Testpad is easy to get started with, especially so if you have existing tests that you can copy'n'paste into Testpad's Import dialog. And if you don't like Testpad, it's just as easy to get your script out as text, and/or CSV if you want results as well.

You can try the interface in the playpen, or sign-up for a beta account and try it for real.

Happy Testing!


Testpad opens its doors; give it a whirl!

$
0
0

After 8 months of building, including several months of limited field trials with forgiving friends, Testpad is now 'in Beta' and ready for feedback from a wider audience. Hooray!

It won't be perfect, that's why we're keeping it in Beta, but it's pretty good and needs some real-world use to put it through its paces.

We're going to keep Testpad free during the beta and then, when the pricing plans start, give a discount to beta customers who've sent us useful feedback.

So come on in; Sign Up for a free beta account and tell us what you think! (OK, not everything you're thinking, just the thoughts that might help us improve Testpad)

Testpad adds iPhone/iPad support for test cases

$
0
0

Big update to Testpad yesterday. You can now 'delegate' testing to your own iPhone or iPad, as well as friends, co-workers and guests. It makes your mobile a remote control for Testpad that let's you pass, fail, query and comment on tests. I've found running Testpad's own tests on an iPad a really natural way to run through a test script, much like holding a clipboard and ticking off items as you go.

Frees up your main display

Perhaps the biggest difference this feature makes is that it frees up the big screen for other duties and cuts down on switching windows every time you want to record a result or read the next test. My iPad (and iPhone) sit idle on my desk a lot, so it seems to make a lot of sense to take advantage of them; they're basically spare input devices.

Seamless handover with QR codes and auto-login

Once a test script is ready for testing (written using a big screen and keyboard), I've tried to make it easy to get going on a mobile. You've basically got two choices: either scan a QR code (a 2D barcode) which instantly loads the link, or email yourself a link that you click on from your mobile's email client. Either way, the embedded link includes an authorisation token, so you don't even have to enter a username and password. It takes you straight to the relevant test script, ready for testing.

Testing Testpad with Testpad

Having the option to drive testing from a touchscreen has always been the plan for Testpad (and hence its name in fact), yet even I was surprised just how convenient it was to use on Testpad's own tests. Testing Testpad with Testpad, juggling test accounts, and remembering which browser is logged in where can all get quite complicated! So it's a huge benefit to put the test script on its own screen (the iPad) and leave the app under test (Testpad, but logged into a temporary account) on the big screen.

Of course, I'm biased. Try it for yourself - it has to be seen (touched? felt?) to be believed!

You can sign up here to give it a try. As always, feedback welcome.

Beta Progress and Latest Updates

$
0
0

Testpad has been live in its free Beta for 5 weeks now and has been getting some great feedback. Testpad's editing speed and the mobile access seem to be going down especially well. Many thanks to those of you that have been sending in your comments and suggestions. Where possible, I've applied that feedback in the update that went live this afternoon.

Updates

So what's changed? Here are the highlights:

  • The plain-text 'source' behind the library scripts are now available on github. It's still very early days for the library of examples, and this is an experiment to see if it makes the submission of edits and whole new scripts simpler. If you'd like to suggest a checklist for inclusion, just fork the 'testpad/library' repository, add the new scripts, and send testpad a pull request with your deltas.
  • Each example in the library now has a discussion thread at end of the script. So if github isn't your bag, you can always make helpful suggestions in the comments instead.
  • There has been some confusion as to when exactly auto-save happens, especially with respect to whether it's safe to navigate off the current page. So to that end, the script-editing page now shows a mini-status in the top left-hand corner which shows one of 'unsaved', 'saving' or 'saved' depending on the current status. I've also added a prompt to warn you if you attempt to leave the page before edits have made it to the server. However, if you've got a half-decent network connection, auto-save should be near instantaneous and this should rarely be an issue.
  • Shift-Enter now makes a new row above the current row; especially useful if you want to make a new first row.
  • The triangles showing open/closed sections of test cases are now clickable. Plus, as before, you can use the spacebar as a shortcut to toggle them open/closed.
  • Some tweaks to the HTTP caching logic to try to ensure the data behind the script page (i.e. the script and its results) are not cached when you use browser-back to come back onto the script-edit page. This seems to behave perfectly in Firefox, IE and Safari so far, with just Chrome being stubborn about loading it from cache.
  • And many small bug fixes. Thanks to those that pointed some of these out!

Next steps

Next up, which you can probably guess if you've been following @testpadapp, is to move towards an official launch of Testpad complete with the pricing plans.

Meanwhile, please keep the comments coming, they all help, however detailed.

Testpad is ready for business

$
0
0

I'm really excited to announce that Testpad today finished its beta phase and launched the pricing plans.

Many thanks to all of you who sent in feedback during the beta. The comments and suggestions have been really helpful in shaping Testpad so far. And please keep your comments coming! Even though Testpad has now launched officially, that doesn't mean we're going to stop making it better and better.

Testpad's pricing plans start at $29/month and are based on the number of active scripts an account is maintaining, rather than the number of users. The idea is that you shouldn't feel at all restricted in how many people you involve in testing, whether it's just one engineer, a whole test team, or a flexible mixture of testers and engineers.

We're starting with the three plans listed on https://ontestpad.com/plans. If you have any questions about how these will work, or if these sizes don't suit your situation, then please send me an email via support@ontestpad.com, I'm sure we can work something out.

UPDATE 19 Apr 2012: Testpad now has a $FREE plan and paid plans starting at $9/user/month in addition to the original script bundle plans

Testpad is using FastSpring's SaaSy service for payment processing. They have a superb customer service team with a great response rate; if you are building an online product and are looking for a payment solution, it's definitely worth checking them out. Also, watch for a longer blog post about my experiences integrating a billing solution.

Meanwhile, good luck with your testing!

Sign up here to give Testpad a try - free.

Checklists. For bringing home Apollo 13. And software.

$
0
0
Image from Heritage Auctions

“Houston, we have a problem” is what everyone remembers. What happened next was Commander Lovell got out a checklist for Lunar Module Systems Activation, pictured left, and used it to ensure Apollo 13 made it safely home - and if you want to own the very checklist he scribbled his notes on, it’s going up for auction on Nov 30th here.

Everyone knows what a checklist is. The diligent (or forgetful) amongst us probably even have one for what to pack to go on holiday. You just can’t argue with their ability to make sure you’ve remembered everything.

When it comes to shipping software, deploying a website or submitting an app to the App store, you similarly want to ensure you’re remembered to check everything. Now this, of course, is the domain of software testing, and many methodologies abound, all sparring for a medal placing on the podium of Best Practice.

But why make such a meal of it?
Why not just whip up and maintain a checklist?

A checklist is really the same as a “Test Plan”, but it just sounds simpler, somehow more approachable. It’s also inherently adaptable to a light or heavy touch when it comes to the degree of control you want to assert or complexity of the software you’re testing.

Checklists can start really simple. Just list the main features that you want to check are working. Then add more detail as time and resources allow; extend it with bugs you missed last time; fold in more detailed scenarios; or concentrate on areas you know are more risky or problematic.

You can treat checklists as a guide to steer Exploratory Testing (formalized ad-hoc testing), as a charter in Session Based Testing, or as a fully scripted Test Plan. You can even layer on a risk model like Google’s recent Attributes-Components-Capabilities approach. Whatever suits your situation.

It was with this thinking that we built Testpad. If these ideas resonate, you should definitely give Testpad a try! The hope is that it will get more projects doing their testing that bit better, and without succumbing to procrastination for fear of having to choose a complicated-sounding methodology.

As Cmdr Lovell appreciated, apart from having to do some trajectory math 200,000 miles from Earth in a cold, cramped, and oxygen-depleted environment, having a ‘simple’ checklist to follow was a lifesaver.

It’s not rocket science.... usually.

Online Payments For SaaS Billing

$
0
0

Testpad recently integrated a billing solution for its subscription plans. With so many Software-as-a-Service products popping up I imagined there would be lots of off-the-shelf options to choose from and it would simply be a matter of fitting our requirements to features offered.

Well, yes and no.

Yes, there are lots of subscription-specific payment providers, but no, integration is not a simple matter. Providers vary in the way they bundle the component elements to take subscripton payments and this has big implications for the design and development effort. Even with a decent all-in-one provider, don’t underestimate the amount of additional work required designing and integrating how the subscription plans will work.


This article is based on Testpad’s experience from integrating FastSpring’s all-in-one solution for our SaaS-based software testing tool. I’m not a payments expert by any means, but hopefully these notes will prove useful to the next guy setting out on this journey.

The Basic Components

Billing for a subscription service basically comes down to a make, buy or ‘assemble’ decision with four main components:

  1. Payment Gateway: the service that performs the actual billing of a customers credit card
  2. Merchant Account: an account into which credit card payments are initially deposited, and back out of which any chargebacks are taken; the merchant account is separate to the company’s bank account (which in turn can be referred to as the ‘settlement account’)
  3. Payment Method Details Collection: the service that collects and stores credit card details and uses them to initiate charges via the payment gateway. As this component handles and stores payment method details it must be PCI Compliant.
  4. Recurring Billing business logic: the code or service that handles subscriptions; the main task of which is to trigger the payment gateway to make charges every billing period. Usually also responsible for handling payment failures, reminder emails (called ‘dunning’), payment holidays, cancellations etc.

With the payment gateway and the merchant account you’ve got little choice but to buy. You don’t necessarily have to buy these separately though as some billing solutions bundle them into a single product. Whether these are bundled or not can make a significant difference to your setup and running costs.

However, payment details collection and recurring billing logic is very much a make or buy decision. Read on for the issues!

What are your priorities really?

Now, before looking at potential payment providers and browsing my list of issues, it’s worth taking a moment to identify what your priorities are. With multiple decisions to make at each turn, knowing what your real goals are will save you time.

For Testpad, our goals had a flavour of Eric Ries’s MVP - Minimum Viable Product Payments. Actually, you could get even more minimal than what Testpad started with and elect to manually bill your first customers until you don’t have time to.

Nevertheless, Testpad’s goals in providing an automated billing solution were to

  1. minimise development time i.e. the amount of UI and code we would have to design, code, test and maintain,
  2. minimise upfront costs such as setup fees or fixed monthly subscriptions,
  3. and by implication: find an all-in-one solution to minimise the points of contact with 3rd-party providers and any integration hassles between different providers.

And non-goals included:

  1. not trying to minimise transaction fees; this will be an optimisation job in the future when there is revenue to optimise
  2. similarly, not trying to minimise foreign-exchange costs (Testpad is based in the UK)

Plus, you might have several further, more detailed, requirements that you would like from your billing solution. For Testpad, whilst we had some in mind (like being able to gift customers a free month if we messed up somehow!), they took a back seat compared to the main goal of minimising dev time.

Comparison of Subscription Payment Providers

There are several companies selling subscription payment solutions. The following are the services that I checked out, offering various combinations of the basic components. All have great reputation at the time of writing. Whilst they also compete on many detailed features, I’m comparing them based on which of the the main building blocks they provide, as for Testpad, these dominated our decision.

ServicePayment GatewayHosted Payment-Details CollectionMerchant AccountSubscription LogicAvailable in UK
FastSpring SaaSyyesyesyesyesyes
Braintree Paymentsyesyou host, lightweight PCI complianceyesyes
Stripeyesyou host, lightweight PCI complianceyesAPI only
Amazon FPSyesoptionalyesAPI onlyneeds a US-based credit card
Recurlyyesyesyes
Chargifyyou host, lightweight PCI complianceyesyes
Spreedlyoptionalyesyes
CheddarGetterdepends on planyesyesyes
PayPal Business Web Payments Proyesyesyessomeyes

Integration Checklist

So. Enough with the basic scene setting. Here comes the detail. All of which is worth considering as it could influence who you partner with and what does or doesn’t get implemented in version 1.0.

In the spirit of Testpad’s testing service, a ‘design test’ for the following list can be found in the example library here: Online Payments Integration Checklist

Who is hosting the payment details pages? (aka PCI compliance)

You will need to be certified as PCI Compliant if your servers receive or store any payment card details. My take on this is that it is generally considered a pain to be avoided.

The obvious way to avoid the compliance process is to not host your own payment processing pages and let your payment provider handle it. Of course, this then means you’re not in direct control of these pages and are reliant on your payment provider for how much customisation is possible.

There are half-way houses too. Most of the providers listed let you host the payment forms, but have them POST their data directly to them and not via your servers, thus avoiding most, if not all, of the process of PCI compliance. With this approach, don’t forget to factor in time to design these pages (and all the other pages that go along with them such as error handling).

With the big impact this makes on how you host these pages, and what it means for the continuity of your branding, it’s worth considering this issue carefully for each of your shortlisted payment providers.

Free Trial

Are you thinking of providing a free trial? How is that going to work – is it free for a limited time, or is it free for a limited feature set?

A big decision is whether to take credit card details at the beginning during initial sign-up, or to require minimal details for the free trial and then look to convert people during the trial. This is probably best answered (in the longer term) with your own split testing. For Testpad, we went with keeping initial sign-up as simple as possible to maximise the audience giving it a try; confident that it will ‘trial well’ and be an easy buy decision later.

Some payment providers support a free first month, but can’t not take credit card details up front. If that’s the case, then you can still do a credit card-less free trial, but you will have to implement that part of the ‘plan’ manually.

Either way, if you’re deferring taking credit card details, don’t forget to design the UI flow and email nudges that encourage conversion during the free trial.

Subscription Plan Design

You probably already know what different subscription options (plans) you want to offer and how they will be priced. If not, that’s something best decided before you continue as it will have a fairly big impact on the payment integration design!

However, don’t forget to consider:

  • are there limits or constraints associated with subscriptions (such as number of users, or other usage metric) that need enforcing in the code?
  • is it even worth enforcing these limits to start with?
  • what is the impact on the UI for subscriptions of different levels? e.g. do you hide unavailable options or show them but with an up-selling message when clicked?
  • subscription period: annual pricing as well as or instead of monthly?
  • special deals for charities, open-source projects, academic institutions?
  • is the product purely subscription based, or are there add-ons outside of the recurring charges?

Chargebacks and Refund policy

Chargebacks are when a customer contacts their card-issuer (e.g. their bank) to query a payment and request a refund. The card-issuer can then forcibly take the payment back out of your merchant account. These can be contested by the various parties involved, and this is a service that some gateways (or integrated payment providers) offer.

If you think your product might have a tendency to incur lots of chargebacks, it’s worth looking into the processes and costs in some detail. On the other hand, if your customer base is less likely to initiate chargebacks, e.g. because you’re selling to businesses and offer a transparent cancellation policy then it’s only worth simply understanding the issue and fees, and not worrying about whether it seems expensive or not (yet!).

Separately to chargebacks, there is also the issue of refunds, where your customers ask you for a refund. All the payment providers have a method for giving a refund, but vary in their fees.

Also consider how cancellations work. Should your customers get a refund for unused days/months of the billing period they cancel within?

Upgrades and Downgrades

If you’re planning to allow customers to upgrade or downgrade their subscription, if that’s even relevant to your product, then consider how that will actually work with respect to the billing cycle. It’s worth checking how your shortlisted payment providers deal with upgrades and downgrades.

Monthly subscription changes are typically simply changing the amount billed on the next cycle, with the new limits/features taking effect immediately. However, annual subscription changes might be more tricky. For Testpad, we just offered what the payment provider offered: the existing annual subscription is cancelled and refunded for unused months, and a new annual subscription is started (and billed) on the day of the upgrade/downgrade.

What UI features will you need to design to handle upgrades/downgrades?

Cancellation Process

How do customers cancel their subscription?

How is the cancellation actually implemented? Don’t forget you will need to effect cancellation both in your own code and with your payment provider so the customer doesn’t get billed again. It would be a disaster if you managed to cancel the account ‘locally’ but left the subscription running somehow with your payment provider.

Does cancellation happen immediately, or does the subscription continue until the next billing date? i.e. when you’re not offering partial refunds, the customer still gets what they paid for.

Do you delete cancelled account data or keep it dormant for reactivation in the future?

Can you prompt to get feedback at the point of cancellation? ‘Exit feedback’ can be very revealing and helpful in improving retention.

Payment Failure Process

Payment failures are very common with credit card subscription payments (expiry, new cards issued, maxed out etc) and customers will need politely chasing up and reminding to enter their new details.

All of the main subscription-centric payment providers cover this, and in my opinion, is a big reason to use a payment provider in the first place - let someone else be good at the code that gets this right!

However, do consider what flexibility you have in terms of:

  • the content of the emails that get sent to your customers
  • the interval and how many emails go out
  • how is a failure finally notified to you and the customer?
  • during the process of chasing up a customer, does your UI look or do anything different?
  • what does your product and the UI do on eventual expiration/cancellation?

Discounts, Coupons and Vouchers

There are many reasons you might want to offer discounts, and the payment providers vary in the models for implementing them, so it’s worth considering what kinds of discount you want to be able to offer and see how they might be achieved with your shortlisted providers.

Consider:

  • granting accounts for free
  • indefinite % discounts
  • N months free or discounted
  • single- or multi- use coupons
  • coupon/voucher codes with customisable text (do they have to be ‘ugly’ strings of random characters)
  • how might affiliate deals work? both in tracking and the nature of the discount they might be able to offer

Migrating Beta Audience

Many services are available as a (free) beta and sign up a significant audience before launching and starting the paid subscription plans.

If you’ve got a beta audience, consider

  • how will these customers be converted into free-trial or paid accounts?
  • will these customers be given a discount?
  • do they get different messages (in the UI or emails) to regular sign-ups?

Currencies

There are three places to consider currencies...

  • the currency your customers are charged in
  • the currency of your merchant account(s)
  • the currency of your settlement account(s)

To keep it simple at the outset, you can just do everything in US dollars. However, you’re likely (but worth split testing!) to see higher conversion rates if you offer prices and charge in local currencies.

Then you’ll need to consider how the price is set in different currencies, as you probably don’t want to list $99 as £63.61; and this might need configuring on both your system (for the pricing plans page) and in the product definitions held by the payment provider.

Aside from the currency choice, Payment Gateways vary in which countries they support in the first place, so definitely check your payment provider/gateway can charge in all the countries you would like to sell to.

Sales Tax / VAT

Depending on your location and the location of your customer, there may be sales tax (or VAT) to account for both in your records and on invoice/receipts sent out to customers.

You might also have preferences for whether your prices are displayed inclusive or exclusive of these taxes.

This is definitely something to checkout with your shortlisted payment providers as their support varies from no help to completely handling it.

Testpad went with FastSpring who completely handle this, so we didn’t need to dig into the implementation details any further.

API Integration (callbacks / notifications)

Before selecting a payment provider, it’s worth looking at (or asking your engineers to look at) how their integration APIs work. For example:

  • how will your system be told about successful subscriptions, changes and cancellations?
  • how are notifications secured?
  • do you rely on receiving notifications or do you also poll subscription status on some pages?
  • how will you test notifications? It’s easy during an initial test mode, but think ahead to when your product and payment solution are live, how then will any changes in code that handle notifications be tested?

And More...

If you’re still reading, here’s a list of further details you might want to check you’ve got covered:

  • frequency of transferring funds from the merchant account to your settlement account
  • analytics and tracking: can you add your Google Analytics and Adwords IDs to the right pages?
  • support for other kinds of payment method, purchase orders, invoicing
  • don’t forget to update Terms & Conditions and Privacy Policies
  • payment provider Test Mode features e.g. simulating payment failure
  • payment provider reporting (dashboards, statements, order lists, statistics etc)
  • process for adding a new subscription plan (and testing it) once existing plans are live? e.g. store test modes

Conclusion

Well, there isn’t really a conclusion as everyone’s requirements will be different.

For Testpad, based in the UK, it was an easy decision to go with FastSpring’s SaaSy service. It is an all-in-one package with a web-GUI to configure products and manage the account; no merchant account setup, no gateway setup, just one company to integrate with.

Had we been based in the US, it would have been a harder decision between Braintree, Stripe and FastSpring as all-in-one offerings. Nevertheless, FastSpring have awesome feedback for their customer service, and deservedly so from my own experience.

If any of this has been helpful, please get in touch. Follow Testpad on twitter: @testpadapp.

And do checkout Testpad if you want a refreshingly simple tool to improve your testing (or if you want to see how well we integrated FastSpring!)


An Interview with Testpad


New features - Projects, Libraries and UI

$
0
0

Testpad saw some big updates today with a whole new UI for grouping scripts into projects. Previously, Testpad had a "manage scripts" page that was a flat list of active scripts. This worked fine for testing single products but became a naming-convention headache for anything bigger. Many of you pointed this out and today, I'm happy to announce the release of this major feature.

So how does it work?

In Testpad, a Script is a list of tests structured as an outlined checklist. Scripts can be used as an outline of functionality to guide exploratory testing, a suite of test cases, or just a list of items you don't want to forget during testing.

A Script also collects its results, which are themselves organised into columns called Runs. So a Script is its tests and the results of those tests.

A Project then, is a set of Scripts, and can be used to make a logical group for tracking and reporting. A Project Report is the collection of reports for its individual scripts. Project reports can be shared internally and externally as required, and as before, both as an interactive HTML page or an HTML-snapshot that can be printed and/or attached to emails whole.

Using Projects to manage releases

To keep things tidy, old stuff can be archived.

Scripts can be archived within Projects, if that makes sense for your workflow, or whole Projects can be archived. This allows for considerable flexibility in how to use Projects. I'm thinking a common convention (and this is how Testpad itself is using them) will be to setup a Project per release of a product. E.g. I've got a project called "Testpad 2.0.1" that is a collection of several scripts covering its different functional areas. When a release is completed, I setup the next release as a new Project (Testpad 2.0.2 say), and copy-forward all the scripts from the previous release. You can copy scripts by dragging them onto their new project with the Ctrl/Cmd key help down.

Copying a script in Testpad has the effect of copying everything except the results, i.e. it treats a Script as an on-the-fly template for a new one. So the new script inherits the tests and the pattern of required test runs. As the scripts evolve with the product, I can edit away in each release, knowing I'm not spoiling the consistency of old tests and their old results.

The old project (release) is then archived as a whole. Report links for archived projects continue to work, so archiving projects just helps to keep things tidy from release to release, without cluttering up the list of active projects.

Also new, libraries of templates

Templates (basically scripts without any results; patterns of tests to be re-used in new scripts) are also grouped - into Libraries. Libraries are just the Testpad equivalent of projects but for templates instead of Scripts. A Template can be dragged onto a project, and a new script will be made from that template (no need to hold down the ctrl key this time).

Libraries of templates are useful for teams who like to pick and choose different sets of tests for different products, e.g. with a customisable and modular product, a subset of templates might be selected from a library and used to prepare a Project that has a set of tests adapted to the set of modules in use.

More control for linking to bug trackers

Projects can have their own bug link pattern, overriding the system default which comes from the account setting. These patterns let you make working links out of the bug numbers in test results.

Big UI changes

The concept of Projects has had quite a bit impact on the management UIs of Testpad, pretty much only leaving the Script-editing page 'unscathed'!

With the right-click context menus and extensive drag'n'drop, it's really efficient to shuffle scripts into place and keep your testing organised.

So check it out if you haven't seen it yet (https://ontestpad.com/) and as ever, please send me your thoughts and feedback: stef@ontestpad.com.

Minor update - expand/collapse controls and more

$
0
0

Small update to Testpad over the weekend, mainly in response to customer feedback... you don't get what you don't ask for! Keep that feedback rolling.

Outline Controls

There are now some subtle controls to expand/collapse the visible outline levels of the rows in a script. You get these controls on both the script editing page and the interactive version of reports. I say 'subtle' as you have to know they're there to spot them...

The options "expand all" and "close all" should be fairly self-evident. As too is opening only to a specified level. Perhaps less obvious is the expand option called "Interesting". This option will close everything and then only open branches that have "interesting" results, where "interesting" is defined as any non-pass result.

More "Visiting" Options

When running a test, Testpad will auto-advance to the next test according to the visiting mode. This release added two more useful modes: visiting everything not already passed, and visiting just results marked as queried.

Performance Enhancements

Lots of improvements behind the scenes aimed at making performance better and more consistent, with the most noticeable effect on big scripts and projects.

And that, with a few minor bug fixes, is that for this update. As ever, please keep the feedback coming.

New Per-User Pricing Plan

$
0
0

Testpad just grew some new pricing plans: a Free Starter Plan (announced in a separate post here) and a Custom Team plan that uses a per-user model as an alternative to the existing "Script Bundle" plans.

Custom Plan, to fit your team size

The new Custom Team plan is simply $9 per user per month with no limit on the number of scripts in the account.

When selecting this plan you can configure exactly how many users you would like your account to support. This number can be upgraded or downgraded at any time. As with other upgrades or downgrades, changes to the account limits take effect immediately, and changes to the amount billed happen on the next billing cycle.

Choice of model: per-user or script-bundles

The new Custom Team plan is being offered in addition to the existing "script bundle" plans (Small, Growing and Large Team). This means you have the choice of having unlimited scripts for a set number of users, or having unlimited users but with a set maximum number of scripts.

Which plan to go for depends on your team size and your Testpad usage. If you've got a big team concentrated on one or two products, you might find the script bundles make more sense. If you've got a smaller team and don't want to be constrained on your usage, the Custom Team will make more sense.

And remember you can upgrade/downgrade at any time, and between all the plan types, so there's no risk in selecting a plan, just try it and if it doesn't seem appropriate after some initial usage, simply go and change it!

One caveat: the Custom Team plan is only available monthly at present, so if you go for an annual script-bundle plan, you can't take advantage of the per-user model (yet).

Any questions or feedback on the new plans, please email me at stef@ontestpad.com.

Simple, astonishingly productive, test management.

New Free Starter Plan for Testpad

$
0
0

Testpad just grew some new pricing plans: a Free Starter Plan and a Custom Plan that's based on the number of users (announced in a separate post here).

As ever with Testpad, these plans were added in response to user feedback, in particular, feedback from smaller startups and individuals who felt the old pricing model didn't fit with their usage model.

FREE Starter Plan

The new Starter Plan is a free plan with full Testpad functionality, but limited to 1 user and 5 active scripts. All new signups start with this plan, which can be upgraded to a paid plan as and when needed (i.e. to enable more users or more scripts).

The free plan is designed for individuals, startups and small teams with only one tester as a cheap (very cheap!) way to get going with Testpad.

The free plan is also intended as the first step for bigger teams to evaluate the what/why/how of Testpad. As and when these teams want to take their evaluation to the next step, a paid plan is needed to enable more users or more scripts - remembering that monthly paid plans can be upgraded, downgraded or canceled at any time, and now start from as little as $9 per month.

Note: For folks on existing Free Trial accounts, these will continue to operate as before, i.e. unlimited usage but that expire when time's up. If your account usage (users and scripts) fit, the subscriptions tab will offer the chance to switch to the new Free Starter Plan.

Any questions or feedback on the new plans, please email me at stef@ontestpad.com.

Simple, lightning quick, test management.

Make your own backups

$
0
0

Although the Testpad servers have got you covered with their hourly snapshots of the database, many customers have asked for ability to download all their data in one go, without having to export each project manually.

New settings tab: Backups

Well, today, Testpad got just such a feature. Account owners (i.e. you need some proper privileges for this one) can browse to the new Backups tab in the settings page, and from there download a ZIP of all the data held in the account.

The ZIP contains folders for each project, with each project containing CSV files per script. Account and project settings are also saved to "info.csv" files in the relevant folders.

The CSV files can be opened in say, MS Excel, or parsed by a script for use in another process.

Automated downloads

If you (or a friendly system administrator) would like to write a script to regularly download your whole account, then enable the "auto-login" link provided. This link includes an authentication token which allows the download without first having to log in.

All optional, of course

But if you don't feel the need, then rest assured Testpad is keeping your data very safe anyway. The database is replicated across multiple servers located in separate availability zones on Amazon's cloud infrastructure. Further, hourly snapshots are archived to Amazon's long term storage service, S3, and regularly tested for consistency.

Good luck with your testing. Any questions, please email me (stef@ontestpad.com)

If you haven't already, give Testpad a try with a free starter plan:

Testpad Home Page

QR codes in web-apps to link desktops to mobiles and tablets

$
0
0

This post highlights a feature of Testpad that I think could be used a lot more across web apps in general - that of using QR codes as a convenient way to move a browser session from a desktop to a mobile. It’s fairly simple to implement and is surprising it's not used more.

The idea is mainly useful for web apps (as opposed to native apps) that have both a big-screen (laptop/desktop) version and a complementary small-screen (phone/tablet) version, and the problem is one of convenience when switching between the two.

Imagine using a web app on your laptop, all logged in and in the middle of some session. Then consider the steps needed to pick up this session on your tablet's browser:

  1. launch or switch to the browser
  2. enter the URL or load the bookmark (requires prior step of making a bookmark)
  3. enter username and password if not cached, remembering that this happens on a touch screen, often prompting with inappropriate spelling corrections
  4. navigate within the app to the item/action of interest

This can easily add up to dozens of taps and tens of seconds, i.e. it's fiddly and can be very frustrating.

Testpad’s solution

Testpad is a web app for writing and running checklists to help with the manual testing of software, especially in agile development environments. It's aimed at teams looking to do better than hacking a spreadsheet/wiki but without going for heavyweight “waterfall-inspired” Test Case Management.


The big-screen UI is for composing checklists with a slick keyboard-driven outline editor. These can then be stepped through by a human on either the same big-screen UI or on the complementary mobile phone/tablet UI.

For Testpad's users, the mobile UI makes for a very convenient way of separating the app that's under test from the app that's managing the test run.

Instead of getting the user of a mobile to login and then navigate to the relevant account/script/test run, Testpad offers a QR code as a shortcut from within the big-screen UI. The QR code contains a URL like:

https://<account>.ontestpad.com/script/2/run/1?auth=<token>
which, when loaded by the mobile browser, takes the user straight to the UI for running those tests.


Now, instead of dozens of taps, the relevant part of the mobile version of Testpad can be reached in as little as one tap - which is the tap required to launch the scanner app (and assuming the scanner goes straight to scan mode, like Optiscan does).

There are more screenshots of Testpad’s tablet and mobile UI if you scroll down a bit on the Testpad home page.

If you want to try this for yourself, take advantage of the free starter plan. Simply create an account, open the “Hello World” script and click on the QR code button as illustrated above.

Implementation Notes

The simplest way to use a QR code in your web app is to include a fixed code that acts as a mobile bookmark. Google search for “qr code generator” for several online services that can generate a static image of a QR code.

However, it’s more interesting if your app can generate dynamic QR codes that are specific to the current context and user of the app.

For several years now, the easiest option here has been to use Google’s Infographics API for QR codes. Simply serve an <img> element with a ‘src’ attribute pointing to this API, with the desired URL encoded as a parameter (along with other options). Sadly, Google have announced the retirement of this service in April 2015 (see the deprecation notice at the top of their page). Nevertheless, it might still be worth using this API for now because its integration is so quick, and replacing it with a DIY solution if the product/feature is proving its worth in a year or two.

Alternatively, for in-browser solutions, take a look at these JavaScript generators:
http://www.d-project.com/qrcode/index.html (renders as an image with a data URL)
https://github.com/jeromeetienne/jquery-qrcode (renders in a canvas tag or table tag)

And similarly for server-side generators, there’s the Java zxing project, or a pure Python library pyqrnative. For the Python library, StackOverflow have a question on serving dynamically generated images in Django which might be helpful.

Security caution: if also encoding an authorisation token in the URL that can automatically log someone in, don't forget you probably want to implement an expiry date for the token. As depending on the sensitivity of the app and its data, this is a magic key that logs in anyone in possession of (or in line of sight of) the QR code on the screen!

For bonus points, the URL you put into the QR code can easily also include tracking parameters for stats and analysis. It's always useful to know if, how, and by whom, your QR codes are being used.

Email alternative

As cool as QR codes are, there is another alternative that doesn’t even need a third party scanning app, and that's having the web app send the user an email containing a link to the relevant mobile URL. Most email clients in mobiles will let the user tap the link and cause the browser to load the page. This is not quite as instant as a QR code as you have to wait for the email to deliver, but it’s still faster than trying to get there by hand. Testpad therefore goes for both methods for transferring to mobile: the QR code or an email link.



So if you haven’t already, check it out on Testpad. The starter plan is free and takes only moments to register for.

Follow me on Twitter @testpadapp


New Video: Introduction to Testpad

$
0
0

The Testpad homepage now has a 2-minute video giving a very quick overview of Testpad. Check it out if you're new to Testpad and want a quick way to see what it's all about.

Or send a link to a friend to save you the trouble of explaining it to them!


For the curious: The video was captured and edited on a Mac using the excellent ScreenFlow app from Telestream, and the audio was recorded and processed with the open source tool Audacity. It's hosted on Vimeo for their high-resolution embedded player, as it really needs to be viewed in HD to see what's going on.


Upgrade to Printable Test Reports

$
0
0

The printable (snapshot) reports now include a list of all test comments underneath the summary grid. The comments are those collected during testing, and can be used for any result type: pass, fail, blocked, query, or even no result.

Quite a few customers have been asking for this one, especially where they're not using a separate bug tracker and want to record (and print) any problems in Testpad.

For now this feature is always on - if there is demand to make these comments optional, please shout and we'll make it more configurable.

This change only affects the printable (snapshot) report that is an all-in-one html file that can be saved and emailed as is, as well as printed. The browseable (live) report continues to show comments when you hover over the comment symbol in a test result cell.

Testpad Release Notes on Tumblr

$
0
0

Detailed release notes and status updates for Testpad are now being published on Tumblr at ontestpad.tumblr.com.

This blog (the one you're reading) will continue to cover major news and updates about Testpad, so you only need to follow the release notes if you're curious about every detailed change to the service.



Bug tracking - multiple issues per result

$
0
0

Testpad has always let you insert bug numbers against results. Typically these come from a 3rd party issue tracking tool such as Jira or Bugzilla. Further, these numbers can be a clickable link to the relevant page in the 3rd party tool if the bug-link pattern has been configured (either in project settings or account settings).

But in an update today, Testpad now lets you enter more than one bug number against one result. Simply separate numbers with a space and Testpad will treat them as separate issues when generating links in both the script-edit view and the various report views.

There isn't much space for more than a couple of numbers to be seen in the script-edit view, so the printable report has been extended to list all the issues against results, again as clickable links, in the comments section.

Copy/Paste Between Testpad Scripts

$
0
0

The Copy/Paste feature in Testpad's script editor has been upgraded to work between scripts. This feature now uses javascript's HTML5 localstorage feature to save the clipboard to a place that survives page loads.

Between Page Loads

This means that tests copied on one script can be pasted into another script even after navigating to another script in the same or a different project.

Between Tabs

It also means that tests copied in a script viewed on one tab can be pasted into a script open on a different tab.

The only restrictions are that it will only work within the same account and on the same browser.

Paste Above or Below

As before, you can paste copied tests above or below the current selection.

By default, Ctrl-V (or Cmd-V on a Mac) will paste after the current selection, or at the end of the script if there is no selection (press Escape to clear the selection).

Shift-Ctrl-V (or Shift-Cmd-V) will paste before the current selection, or at the top of the script if there is no selection.

Copying a row will automatically include all the child rows (indented rows) of the current selection, making it very convenient to select and copy significant portions of a script in just a few clicks.

DIY Cut Support

Ctrl-X remains unsupported, but is simple to achieve by first copying with Ctrl-C then deleting with Del or Backspace. The copied (now cut) rows remain in the clipboard after the deletion and can be pasted back to the same script or a different script after navigation or change of tabs

Browser Compatibility

HTML5 localstorage is well supported these days, so it should work in the major/modern browsers. We have tested the new copy/paste on IE9/IE10 on Windows7/8, and Safari/Firefox/Chrome on OSX, but if you have any issues, please get in touch - stef@ontestpad.com.

Keyboard Shortcuts for Even Faster Testing

$
0
0

Testpad now supports keyboard shortcuts to set the results of tests during testing.
When running a test, with the Test Run dialog open (pictured right), you can press:

  • 'p' or 'space' for pass
  • 'f' for fail
  • 'b' for blocked
  • 'q' for query
  • 'x' for exclude

You'll need focus on the result buttons (also as pictured right) for these new shortcuts to work.

If you want to record comments and issue numbers, hit 'tab' to cycle through the input boxes and finish a test by setting its pass/fail status. For example, the sequence to fail a test with a comment and issue number would be:

  1. Press 'tab' to move focus to the Comments field and type a comment
  2. Press 'tab' to move focus to the Issue field and type an issue number
  3. Press 'tab' to move focus back onto the result buttons
  4. Press 'f' to fail the test and auto-advance to the next test

Really Fast Checklists

If your tests are just simple checks that mostly pass without comment, then the spacebar shortcut (as an alternative to 'p') makes for a very fast way to zap through a checklist, ticking off items as they're checked.


Edit Tests during Testing

As before, you can continue to edit test decriptions during a test run. This is useful as you're most likely to discover a problem with the test description during a test run, and being able to edit the test right there and then saves on having to remember to come back later

Using the mouse, you can click on the test text, make the edits, then click back on the Test Run details dialog to continue testing.

Or if you prefer to stay on the keyboard, hit 'Esc' to close the Test Run dialog, hit 'Enter' to start editing the currently selected test row, make the edits, hit 'Esc' to finish editing the test row and type 'Alt-X' to resume testing.


As usual, if you've got any comments or feedback on the new shortcuts, please send me an email at stef@ontestpad.com.

Viewing all 42 articles
Browse latest View live