Four considerations for evaluating GenAI for femtech

As more industries explore the potentials of GenAI innovation and efficiency, it’s helpful to have a roadmap toward successful implementation. Investing in GenAI solutions can be costly, so it’s critical to approach the evaluation process strategically.

Planned Parenthood Federation of America (PPFA) is proudly committed to applying technology and innovation to ensure people can get the sexual health information they need to seek the care they deserve.

Over the years, we’ve learned a lot about innovating with an eye toward equity, even with finite resources.

In addition to investing in our digital tools, one of our top innovation priorities is strategically researching and evaluating GenAI to explore its potential to help us continue to meet our organisational goals.

Here are some suggestions based on insights we’ve learned along the way: 

1. Determine “Why GenAI”

Test customer support as a business case.

Consider areas where automation can significantly improve efficiency or customer experience.

Common use cases like customer support and FAQ handling are a good place to start. Information sharing with motivated users is often the strongest tool to embolden positive action or change.

Define success metrics.

Make sure the product has clear, measurable objectives for chatbot implementation.

Objectives could include reducing staff time or cost or increasing satisfaction scores.

Start small and iterate.

Attempting to build a comprehensive chatbot that can respond to any scenario can result in a product that needs to be more specific and defined, making success more difficult to measure.

Instead, consider investing in AI that is trained for a specific purpose in one area. With increased specialization comes optimal output tailored for your audience.

2. Prioritise Diversity, Equity, and Inclusion (DEI) for Bot Success

Ensure inclusive language and cultural sensitivity.

A successful chatbot — particularly in health care — should be able to communicate effectively with anyone, acknowledging and accounting for diverse lived experiences, and be quality tested to ensure a reduction of error or removal of bias.

Build a diverse development and review team.

Make sure the product is regularly staffed with and work reviewed by a diverse (representative in race/ethnicity, age, lived experience, for example) group of people to contribute to the bot’s accuracy, fairness, and inclusivity.

3. Ensure Human-AI Compatibility Throughout the Product Life Cycle

Invest in resourcing staff for ongoing development and training for the bot.

Implement safeguards to prevent the AI from producing harmful or inappropriate content by handing it off to humans — or even another system — if something goes awry or the user opts out. 

Ensuring seamless human-AI compatibility is crucial at every stage of a GenAI product’s lifecycle.

This ongoing focus on the human element helps create a more effective, user-friendly, and trustworthy AI solution.

4. Score for Accuracy and Brand Representation

Prioritise continued accuracy assessment.

Eliminating inaccuracy in your GenAI solution maximises user trust, builds legal compliance, and achieves business objectives.

To evaluate and enhance the accuracy of a GenAI solution, you can implement mechanisms for users to report inaccuracies or provide feedback.

Systematising error monitoring and implementing solutions for GenAI knowledge cutoff dates can also improve accuracy.

Align your product with your brand.

For an optimal user experience, you’ll want your GenAI solution to reflect your brand accurately.

Think about developing an evolving scorecard that measures communication tone, style, and ethical standards and principles against your brand guidelines and values.

Be sure to include crisis scenarios, compare these scores against other language model technologies, and keep a tally of the number of times AI provides a satisfactory answer based on the agreed-upon definition of accuracy, reliability, and other quality metrics.

If you keep your goal finite and specific, include a representative community of users, and hire trained staff to monitor and evaluate the product, you’ve established the parameters for building, scaling, evaluating, and iterating a successful GenAI solution.

Planned Parenthood is the headline sponsor of the Femtech World Awards. Find out more and enter for free here.