The Art of Quality Assurance for CMS

Quality Assurance is the art of ensuring the quality standards meet the product specifications. Most organizations work towards a ‘Data oriented’ model where the decisions are driven by data, user/customer experience. Quality testers ensure the product quality is met based on the user experience testing approach. Testers used to be techies, rooted in analytical and logical reasoning. They were/are proficient in executing scripts, more systematic, and detail-oriented. Over the changing needs during the new pandemic era, it is becoming more and more important to add core skills related to the knowledge of the sector, business activities, and the social environment. To achieve these goals, testers adopt process-oriented methods to accommodate leading business changes in the new style of work. This ultimately enhances the relationship with the customer and promises satisfaction. 

Challenges in proficient test execution

• Testers need to step into the users’ shoes and anticipate design-driven scenarios that take advantage of new technologies and automation to derive more efficient services which make life easier and allow users to focus on value-added activities. Using a plethora of testing types to match the stage of your product’s maturity will help you scale continuously. 
• In theory, testing requires a functional requirement or the product definition to understand the customer needs and use cases prior to testing. Testers drive towards changing testing requirements and apply techniques by adopting new methodologies to tests at an early stage to detect defects/issues early in the testing life cycle.
• While in the ‘Time to Market’ model, the challenges hike up to timebox the deliverables and inspection. Obtaining high-quality testing by defining use cases at the beginning of the testing life cycle from the product definition and providing a happy path to the end-user for enhancing customer satisfaction is the priority of QA. It typically begins from the time the product is defined and collaborates with the backend/frontend development teams to ensure issues are detected early in the software development life cycle. This is modern-day testing, that has been evolved along with the product evolution to assure quality is met before it goes out to the consumer. 

Web Application Testing for CMS sites

1. Content on Device Test  - Component test, Content types, etc
2. Design test - UX/UI 
3. Functionality test - End To End functionalities; form submissions, email delivery, etc,
4. Cross-browser testing - testing across different browsers
5. Performance/Load Testing - testing the site rendering speed and reliability

Content testing is testing if the right content is displayed on the site built out from CMS.. Contents can be of different types; images, text, images with overlay text that renders on different device types and browsers.  While testing the content, we might find components misplaced, images shrunk, texts with different styling, and background colors mismatched with the original. Testing is to ensure the UI elements work as expected across browsers, devices, screens of different resolutions, OS, versions, etc, Headless CMS solves those aspects substantially. 

The CMS framework is designed with content models to have all different content types in one place in your own frontend language. This dynamically enables rendering in multiple devices with different screen resolutions from a single building architecture. 

Possible defects/bugs to foresee would be:

1. Component disposition on the UI
2. Web Element design OFF 
3. Functionality of button, images, links, and the desired landing pages
4. API call failures, HTTPs response errors, server responses
5. Missing content types, title, description, etc.,
6. Image resolutions on different devices 

Testing methods play a vital role in detecting defects early in the software life cycle. Although manual testing involves more White-box and intermittent Grey-box with partial knowledge of the code, the White-box method gives more perspective to code analysis in an early phase at the development level. These methods are applied in various stages of testing till the integration happens. Once the backend and front-end integration testing pass, manual testing and automation come into the picture. This entire life cycle further reduces the cost to fix defects later in the product life cycle. 

While testing is a vast subject with various techniques, CMS sites need a different approach. 

Technically, any UX/UI design engineer would look at the color, style, background image, and the overlay of images, text format on the site. Looking at the button elements, image components, the framework of the UI elements having a border, adding style to it is all about design. Testers should have a similar artistic eye to match the customer's need for designs to deliver a better product. QA works collaboratively with the Full Stack developers to ensure the design aspects are met as per the requirement. This is all about Front end testing.  

• 
  

The look and feel of the site is a done deal, while functionality is always a question the Dev team has. Does the button work when clicked? Does the search query bring in results as expected? Does the brand logo land on the Home page and so on...How does a site interact with the user? It is the end-to-end functionality of the elements on the page. All web elements are significant to the tester for an end-to-end transaction to complete. 

This is something that a Front end tester would look at in a functionality test. What if the About link doesn’t land on the ‘About’ page. It means, either the link is broken or not configured yet. Another functionality test that would be apt in a CMS site would be query results from search. The content lookup should draw the desired data from the query. The data comes in through the CMS when it is integrated with its desired apps like Cloudinary, HubSpot, etc. If products/images are added into the content models and published, they render on the UI without any delay. When there is a delay to render an image related to the content, it affects the site's performance which results in possible failures in loading enough traffic on the site at a given time. 

*[Note: Backend testing plays a crucial part of the testing life cycle which involves QA before it shifts to the front end. I have covered only Front End Aspects in this blog]

• 
  

Interestingly, we have browsers that evolve from time to time, and apparently, the world uses some browsers by default due to their reliability and performance in the system/device. The most commonly used browsers are Google Chrome, Edge, Firefox, Safari, etc., All of these browsers have their unique capability of rendering sites to the level of user interaction. Having an eye for possible browser issues while rendering content is one important aspect of QA. Every browser has its way of displaying images, color, and positioning the UI elements. Although we have a couple of browsers sharing a similar platform(chrome and Edge), the speed and performance of the site vary on each OS and its version. 

Modern browser-based applications act as both a friend and an enemy to the tester when the site interacts. Some unique browsers have the capability to stop rendering content or links if it is configured to protect the user’s privacy and security. They could be an enemy to the tester in this case as the site fails to perform or give its best. While enabling or customizing some settings in those browsers would be a big win-win to the tester with possible solutions to various issues on the site. 

Any third-party extensions also behave weirdly in CMS sites at random times of the day. Although extensions support speeding up the testing process in an agile model, it also has their own limitations to display content as desired.

Quality Assurance and its non-tech savvy people

Our CMS sites are more efficient in serving non-tech-savvy users by possibly solving their use cases to reduce the cognitive load. User-defined usability testing comes into the picture while testing different categories of users with little to no knowledge of the site. Even after meeting the acceptance criteria from the customer, the end-user experience can always be daunting to predict. The actual use cases are very different from the internal code and how it behaves in the front-end architecture. Quality testers act as a significant bridge in articulating the desired end-user experience in a real-world scenario for a product that can go to market in a very short period of time.  

In conclusion, Quality Assurance in CMS sites is the most challenging among all types of testing and when done with a detail-oriented mindset, the strategic approach enables the tester to detect defects early, usability issues, security concerns, and so on to ensure the quality of the product is met before it reaches the end-user/customer. 

Written by: Arthi Morekonda Quality Assurance Engineer @konabos Inc.