Planning and Delivering an AI Project Successfully
by Daniel Lambert (book a 30-minute meeting)
​
According to the Harvard Business Review, 80% of AI projects fail to deliver on their promises[i]. This high failure rate is attributed to a variety of factors, including a lack of clear business objectives, insufficient data quality, and the complexity of integrating AI solutions into existing business processes. To achieve tangible results with your AI projects, follow these 24 essential steps encompassing strategic planning, project planning, development, deployment, and post-deployment phases.
​
Strategic Planning Phase
​
To succeed in delivering your AI project, it's imperative to execute thorough strategic planning. This phase involves several critical steps: 1. identifying a business opportunity, 2. defining business objectives and scope, 3- mapping your business architecture, 4- identifying the required information, 5- designing the data architecture, 5- defining your system architecture, 6- improving and automating processes, 7- selecting the appropriate AI model, 8- establishing the appropriate security architecture, 9- building the project roadmap. Finally, you must decide whether to proceed further based on these foundations.
​
1. Identify Opportunity
Examining market research, industry analysis studies, and analyzing trends and emerging technologies is important. However, to ensure your project's success, your primary focus should be on identifying the problems and needs of your customers, partners, and other key stakeholders in your industry. Opportunities often arise from collecting and analyzing feedback from current and potential customers and partners through surveys, interviews, and focus groups. By identifying pain points and common problems with existing products or services, you can uncover profitable opportunities, whether they involve AI or not. In brief, you need to focus on your most valuable opportunities with the quickest time to value. Once an opportunity is identified, properly evaluated, and you have secured a business sponsor, you can then proceed with the detailed planning of your AI project.
​
2. Define Objectives and Scope
Once you have identified an opportunity and secured a sponsor, it's crucial to clearly articulate the project's objectives and desired outcomes. Follow the 7 phases of strategy mapping to ensure comprehensive planning, as shown in Figure 1 below. Additionally, clearly define the project's boundaries, specifying what will and won't be covered.
3. Business Architecture
After identifying your project's objectives, the next step is to develop a business architecture. This involves describing the value proposition offered to the triggering stakeholders (such as customer segments, partners, or personas) involved in your project. You should map the value stream that will deliver this value, identify the enabling level-3 or level-4 capabilities of this value stream, and finally, determine the participating internal and external stakeholders involved in providing value. For more insights, you may want to refer to this article entitled “Using Business and Enterprise Architects to Increase the Success Rate of SAFe® Projects”.
4. Required Information
A crucial part of your business architecture is identifying the required information of the value stream that is delivering the necessary value proposition. Gathering and storing data is easy. Making sense of data and extracting information from it is another story. What portion of your data can actually train artificial intelligence in such a way to provide value to users and customers is much more difficult. In brief, what valuable information can we extract for our data?
Most business and enterprise architects understand business capabilities and their supporting applications. This is not enough. To build a modern data architecture, business and enterprise architects should also understand the need to examine information concepts (or information type or business objects). Business and enterprise architects need to ask themselves what information is required to deliver a value proposition to a client or a user with a value stream as shown in Figure 2 above.
​
Business and enterprise architects also need to identify what information can be created, modified, and/or used by business capabilities and that are stored in one or several databases as shown in Figure 3 below. Having reliable data sources is crucial for developing valuable artificial intelligence models. High-quality data ensures accuracy, reduces biases, and enhances the model's ability to make accurate predictions. Robust data allows for better training, testing, and validation of AI algorithms, ultimately leading to more effective and trustworthy AI solutions. Investing in good data sources lays the foundation for building models that can drive meaningful insights and innovations in various fields.
5. Data Architecture
​
To deliver a successful AI project, an organization needs a modern data architecture that is agile, quick, flexible, and easy to implement. Such an architecture enables the organization to efficiently locate and utilize the necessary information that provides value. Without a proper and contemporary data and information architecture, the chances of success for your AI project are significantly diminished, as formulated in this article entitled “The Importance of a Modern Data Architecture for Successful AI Projects”.
​
The data architecture of your project will involve the design of a data storage solution that ensures data integrity and accessibility and the mapping of the data flow from collection to processing, to storage, and to the model training of your AI project. Data architecture also involves governance and the implementation of security measures to protect adequately sensitive data.
​
If most of your historical data is unavailable for your AI project, you will need to generate this data or acquire it from external sources. This can substantially increase both the duration and cost of your project. Creating data involves designing simulations or experiments to produce relevant data, which can be time-consuming. Alternatively, purchasing data from third-party vendors might provide a quicker solution but can be expensive. Both approaches require careful consideration of data quality and relevance to ensure the success of your AI project, as well as potential impacts on budget and timelines.
​
6. Process Automation​
Delivering a successful AI project involves enhancing and automating an organization's current processes. Process automation begins with analyzing and documenting existing workflows through process mapping and identifying bottlenecks. Next, a high-level design of the automation solution must be developed. This includes selecting the appropriate automation tools and technologies (e.g., RPA, AI, ML), described in section 8 below, and designing the automated workflow, detailing each step and decision point to ensure seamless integration and functionality.
7- Requirements Elaboration
To enhance the success of your AI project, it's essential to craft detailed valuable business requirements through the lens of business architecture, as illustrated in Figure 4 below. The requirements identified in step 7 will be further refined, elaborating on their epics and user stories in step 14. This detailed approach ensures clarity, alignment with business goals, and a solid foundation for project execution.
8- AI Model Selection
​
There are many AI models available, and enterprise architects must select the right one from supervised learning models, unsupervised learning models, reinforcement learning models, and deep learning models. These models enable diverse applications across various domains. Enterprise architects should choose a suitable AI model and architecture based on the specific requirements of the AI project. The chosen model must also be customized to fit the project's unique needs. Finally, to assess the model's performance and effectiveness, appropriate technical evaluation metrics must be defined and used for evaluation.
​
9- System Architecture Design
​
With your business and data architecture in place, you can now design the overall system architecture of your AI project, emphasizing scalability, reliability, and performance. IT architects will define system components such as data pipelines, model training, and deployment infrastructure. They will also identify integration points with existing systems or third-party services, utilizing appropriate APIs to ensure seamless connectivity and functionality.
​
10- Talent and Skill Development
​
Your organization needs to build a valuable AI team. This does not come cheap. As shown in Figure 5 below, it should include the following roles: AI Architect, Engineers, Model Validator, Business Owner, Business Expert, Data Engineer, Data Scientist, and AI Expert.
​
Trying to hire most of your team members outside your organization will most probably take too long. Using trained internal resources for most of these roles will accelerate delivery significantly. This is why you will need to invest in training and upskilling your AI team to keep up with the latest developments in the field.
​
11- Project Roadmap
​
Before starting the AI Project planning phase, enterprise architects will need to elaborate a project roadmap that will outline the strategic plan, key milestones and business outcomes, timelines, and deliverables of the project. It provides a high-level overview, guiding the project from initiation to completion, ensuring alignment with objectives, resource allocation, and stakeholder communication. The roadmap helps manage expectations and track progress throughout the project lifecycle.
​
12- Go or No Go?
​
At any time between steps 2 and 14, a decision can be made to halt the project based on its feasibility and viability. During strategic planning, you might discover that market demand is lower than initially anticipated or identify technical challenges that make the AI project infeasible. It's crucial to assess the financial viability of the project, evaluating potential return on investment, cost structures, and profitability. This ensures informed decision-making, allowing for project termination if it no longer aligns with business goals or proves too risky to continue.
​
Project Planning Phase
​
Once your strategic planning is completed, you need to proceed with the details by understanding more your data, selecting the right tools and technologies, and developing a detailed project plan complete with epics and user stories.
​
13- Understand the Data
​
In your project planning, it will be essential to gather all necessary data, ensuring its relevance and high quality. Additionally, you must clean, preprocess, and format the data for AI model training. Finally, ensure compliance with data privacy regulations and ethical standards to maintain integrity and trustworthiness throughout the project.
​
14- Select the Right Tools and Technologies
​
When detailing your AI project planning, it is crucial to select the appropriate AI frameworks, libraries, and hardware, as guided by step 8, the AI model selection. Additionally, accurately establishing the computing infrastructure needed for development and deployment is essential, based on step 9, the system architecture design. This ensures optimal performance and scalability for your AI project.
​
15- Develop a Project Plan
​
Project planning entails developing a comprehensive plan, which includes detailing epics and user stories from the requirements identified in step 7. This phase also involves creating a detailed timeline with key milestones and deliverables already mentioned in step 10, the project roadmap. Identifying potential risks and developing mitigation strategies is crucial. Additionally, the careful selection and assignment of human resources to various project phases are critical elements for ensuring the success of your AI project.
​
Development Phase
​
Once the sponsor has accepted the AI project plan, the development phase can start. This phase includes model development, iterative improvements, integration, and testing to ensure scalable delivery, as explained in this article entitled “How To Architect And Deliver AI At Scale”. The development, deployment, and post-deployment phases involve various AI stakeholders, as depicted in Figure 5 below, that are crucial for a successful implementation.
16- Model Development
​
The development phase begins with the AI model development, where the appropriate algorithms for the AI tasks are chosen and automated model training pipelines are implemented to streamline the process. The development team will train the AI model using the prepared data, as outlined in step 5, and iterate frequently to enhance the model's performance. Regular evaluations using relevant metrics are essential to ensure the model meets the performance criteria (including drift indicators) and business outcomes defined in step 2 (business architecture).
During the development of your AI model, ensure the creation of user-friendly interfaces for both data scientists and non-technical stakeholders, such as dashboards and APIs. Additionally, focus on delivering AI outputs that are accessible and easily interpretable for end-users.
​
Your AI model development also needs to prioritize AI system security to protect against threats and vulnerabilities. This can be accomplished by regularly updating and strengthening your AI software components to mitigate security risks.
17- Iterative Improvement
​
During the development phase, frequent iterations will be conducted to improve your AI model. This includes continuous testing on validation datasets and incorporating regular feedback from participating stakeholders, as identified in step 3, to refine the models accordingly.
18- Integration and Testing
​
The development phase of your AI model will end with its system integration and testing. The development team will need to integrate the AI generative models with existing systems or workflows of your organization based on your system architecture designed earlier in step 9. The integration of your AI model will also require the development team to conduct comprehensive testing to ensure the models work as expected within the larger system.
​
Deployment Phase
​
Once the testing of your integrated AI model is complete and satisfactory, you will be ready to deploy.
​
19- Deployment Preparation
​
To prepare for the deployment of your AI model, your organization will need to plan the deployment approach, whether phased, full-scale, or beta testing.
​
Ideally, the deployment of your AI model should be as microservices or serverless functions to make them easily accessible to applications and users. A robust, scalable, secure deployment of your AI model will be based on a cloud-based computing infrastructure to support AI workloads. Cloud services like AWS, Azure, and Google Cloud provide the necessary resources. AI model deployments are most efficient when leveraging containers and orchestration tools like Docker and Kubernetes.
​
Deploying your AI model requires establishing monitoring mechanisms to track performance and identify issues post-deployment. This ensures ongoing functionality and allows for prompt resolution of any problems, as shown in step 22 in the post-deployment phase.
​
Before deploying your AI model, it is crucial to prepare for change and ensure all users are adequately trained as you transition from pilot to production. This includes providing comprehensive training sessions to familiarize users with the AI system, its functionalities, and best practices. Additionally, establish a support system to address any issues or concerns that may arise during the transition. Proper preparation and training will facilitate a smoother adoption process and enhance user confidence and proficiency with the AI model.
​
20- Deployment
​
Deploy your AI model to the production environment only after thorough preparation. Ensure that your IT infrastructure is capable of scaling to handle the increased load. This includes verifying that the system can manage anticipated traffic, data processing, and computational demands. Additionally, confirm that monitoring and support systems are in place to promptly identify and resolve any issues that may arise post-deployment. Adequate preparation and infrastructure scalability are essential for the successful and sustainable deployment of your AI model.
​
Post-Deployment Phase
​
The journey of your AI model delivery does not conclude with its deployment. The model will continue to learn and improve, but it may also experience sometimes issues like hallucinations or biases. Therefore, it's crucial to continuously monitor and maintain your systems, making ongoing improvements after the deployment of the AI model. Your AI project will only be considered complete once you have generated comprehensive documentation, implemented necessary updates, and conducted a thorough post-project review.
​
21- Monitoring and Maintenance
​
After deploying your AI model, it is essential for your organization to continuously monitor its performance and system health. Promptly address any issues or anomalies that arise to ensure optimal functioning by setting up proper alert mechanisms. Regular monitoring, as shown in Figure 6 below, helps detect potential problems early, allowing for quick resolutions that maintain system reliability and accuracy. This proactive approach ensures that your AI model continues to meet performance expectations and contributes effectively to organizational goals.
22- Continuous Improvement
​
To ensure continuous improvement of your AI project, regularly gather feedback from end-users and stakeholders. Also update your AI model as needed to adapt to new data and changing requirements, ensuring it remains effective and relevant.
​
23- Documentation and Reporting​
​
A project is incomplete without proper documentation and reporting. You need to maintain comprehensive records of the project, including model design, data sources, and deployment procedures. Regular communication is equally important; provide therefore consistent updates on project progress and outcomes to stakeholders and the sponsor.
​
24- Post-Project Review
​
Your AI project will only be considered complete after your organization has conducted a thorough post-project review. This will include the analysis of what went well and the identification of areas for improvement during the planning and delivery phases. Documenting lessons learned is also crucial to inform and enhance future projects.
​
​
By following these 24 essential steps to plan and deliver your AI project, you significantly increase the likelihood of meeting your objectives and achieving tangible results. Each step ensures thorough preparation, strategic execution, and continuous improvement, paving the way for successful project outcomes.
​
__________________________
​[i] Source: “Keep Your AI Projects on Track” article written by Iavor Bojinov in the Harvard Business Review Magazine in November 2023.