Project Goal
Improving accuracy of the company’s current AI translation models through English informaitons translated by users.
3 weeks
Project duration
Human resources: Only me and one intern.
Personal energy: I have to work on two other projects at the same time.
Time constraints: only 3 weeks
constrain
my role
Led the MVP design process, incorporating insights from MVP performance metrics to enhance functionality.
Conducted user interviews and developed user journey maps to capture in-depth user insights.
Collaborated with engineering teams to define and refine requirements for the translation features and achievement system.
Directed the UX design initiatives, establishing the design language based on emotional cues, spearheading usability testing, and coordinating team tasks.
DAU/MAU
2.6X%
2.9X%
+17.2X%
Translation utilization rate
Translation modeling accuracy
outcome
Project overview
technical background
NLP (Natural Language Processing) technology is used to make products that face normal users. "XiMei" mainly uses one of the translation models (translation BLEU score over Google by 40%)
Product Journey Maps
Project background
Data Crawling
translate
Accessing the database
check
release
Editors get tons of news and articles from abroad
Translation using the company's translation model
Tag articles and add to database
1. Machine review: Screening based on the sensitive words provided by the government;
2. Manual review: screening and reviewing the articles that have been reviewed by the machine.
Posted manually by editors
Present problem
technology
Present problem
solution
Resource allocation/progress
Application
Optimize translation algorithms
The accuracy of the translation model has not been significantly improved for a long time, resulting in uneven accuracy, fluency, and expressiveness of the translated text.
Not achievable in the short-term
Increase translation model complexity
Insufficient technical support
Increase the diversity of the corpus
Ongoing and slow
Human intervention and correction
Insufficient human resources
Technologies
translation model
Chinese-English bilingual article
translate
read
Can we involve the users and do manual intervention?
users
Minimum Viable Product (MVP) Exploration of Translation Features
Develop business hypotheses and conceptualize MVP scenarios based on business objectives
Setting business goals
Developing business hypotheses
ideation and solution
Measurable indicators
Improve translation model accuracy through human intervention by users
Encourage users to modify and optimize the existing translation corpus by giving them certain cash rewards to provide user-friendly, precise and smooth translations. Improve the accuracy of the translation model and raise the company's technical barriers.
Users' participation in the translation process also improves daily activity and user retention rate.
Users are recruited to participate in translation tasks through a company-run social media network, avoiding potential competitors from detecting the company's strategy.
Rewarded with $1 per translated article.
Number of participants
Number of translated articles
Accuracy after corpus modification
implement
1~2 interns will working for article export and daily maintenance of the translation community
Recruiting users into translation groups on their social media groups
Interns need to select untranslated articles in the backend and batch export them; post the original articles in the translation group, and users receive the translated articles and translate them.
After the user has translated, he/she submits the task and at the same time gives feedback in the group (which can motivate some of the users), giving a cash reward of 1 $USD.
The translation will be submitted to the editor for review, and after the review is completed, it will be imported into BLEU with the original translation for scoring and results.
Recruitment of translation users
Posting translation assignments
Submitting translation assignments
Review, rating
user needs
From the user's point of view, translate business goals into user behavior, guide users to use the translation function, and help the product achieve business goals.
Quality metrics
Translation participation rate
Translation model accuracy
motivations
Improve users' English skills by correcting mistranslations
Looking for someone to communicate and interact with
How does it work?
The translated text is too long.
Guidance for beginners
Change whole article translation to paragraph translation
Provide users with the ability to modify the translation by overriding the original translation with the approved translation.
Allow other users to give feedback (comments, likes, etc.).
What is the quality of the submitted translations?
How to evaluate other people's translations?
Is the translation too difficult?
How do users interact with the community and meet friends?
Is there room for improvement in participation?
Does it take too long to review translations?
Users can vote for the translation to be displayed.
Provide feedback function
Free selection of translated paragraphs
Add social features such as likes, private messages, etc.
Design user motivation model
Provide feedback within 2 hours
obstacles
opportunities
Users use the translation function
Typing translations
Submit translations
Get feedback
User behavior
Key factors analysis
Solution
MVP test results
47.4%
Participation rate in translation
Number of participants 83/175
Confirmation by editor
Number of translations
Bonus incentives for translators
corpus pass rate
160+
$110+
72%
In multiple BLEU evaluations, the human-modified corpus scored higher than the machine-translated corpus
Translation Model Accuracy Improvement
12.32%+
design strategy
Understandability
Fluidity
Engagement
Users can easily understand translation features and how to use them, avoiding confusion or misunderstanding. Reduce the pressure on users to understand how to use translation features.
Reduce interruptions or pauses in the user's translation experience through design methods such as interaction, feedback, and visual hierarchy.
Increase the frequency of user participation and the amount of translated content through emotional and material incentives.
Engagement
Goal: Encouraging user participation in translation
Why should users be encouraged to participate in translation?
Improve the accuracy of translation models by providing users with easy-to-use, fluent, and accurate translations
Improve translation quality
Increase user engagement, loyalty, and social interaction
Analysis of existing user motivation methods
Brief
Usage Scenarios
Feature
Collect points based on the user's behavior in the product and provide rewards and benefits based on different point levels and points.
Diversified rewards, operational flexibility, high user engagement
E-commerce system, flight miles Points, games, etc.
A rewards system that incentivizes users to participate and use the product by setting specific tasks and goals for users.
Results-oriented, encouraging interaction
Delivery software, social media
Incentivize users by offering them membership privileges and benefits
Member Privileges, Personalized Recommendations, Member Levels
All kinds of VIPs
A way to incentivize users to participate, complete tasks, and achieve a goal through rewards.
Goal-oriented, immediate feedback, sustainability
Exercise products, online learning products, games
Attract users' participation and support through the product's contribution to environmental protection, public welfare, etc.
Combining social responsibility and business benefits
Community apps, NGO's, public service promotions
Points System
Task System
Membership System
Achievement System
Social-Responsibility System
Interviews - users' needs
Through user interviews, 4 user needs were summarized: learning needs, self-presentation, sociable needs, competitive desires.
Users improve their English skills by correcting incorrect translations or translating English texts in preparation for taking English tests.
Getting feedback, having someone to talk to.
“While reading the articles translated by XiMei, I saw some translations that were strange. I often asked these questions in the user group and later knew it was machine translated. I joined the translation group during user testing to help correct this app and improve my English at the same time.”
“As an English major, I have to listen to the news and translate it every morning. This feeling of having no one to communicate with is boring.”
“XiMei has a great selection of foreign articles. Although they are available in both Chinese and English, I often read only the original English text while translating it in my mind and then read the translated article for comparison. This helps me better understand the logic and structure of the English articles. Although it is slow at the beginning, I think it helps me to improve my English skills.”
“In order to get more practice and feedback, I have joined some English learning communities before. I feel that my English has improved a bit and I have become more confident.”
“When I practice IELTS reading, I have to translate the articles and I think it's a great opportunity to compete. I like to compare myself with other people whose translations are of higher quality. I don't need a reward, I need the honor.”
“I am not good at writing in English but don't know how to improve it. Since exam courses are expensive, I joined some writing communities where other members evaluate my writing and give me some feedback. Through communication, my writing skills improved and my self-confidence improved at the same time.”
-Junior in Economics, IELTS student
-First-year graduate student, English major
-Junior Architecture, IELTS student
-Sophomore in high school, student
-Fourth year university student
-Junior in college, IELTS student
Learning needs
sociable needs
Users are eager to show their English skills and increase their sense of confidence.
Gain a sense of victory and honor by competing with others.
self-presentation
competitive desires
Users
Needs
Engagement method
Combining the achievement system with the existing task system can better motivate the target users to meet the demands of learning, socializing, self-presentation, and other aspects.
Learning needs: improve your English by correcting incorrect translations.
Self-representation: to show one's translation ability and increase the sense of self-confidence.
Social needs: to get feedback and have someone to communicate with.
Competitive desires: to gain a sense of victory and honor by competing with others.
Most of the target users are students who need to take the IELTS and TOEFL exams. Most of the articles in these reading sections come from the media, magazines and journals. By reading articles from the same sources in XiMei, understanding the terminology as well as collecting the original texts and Chinese translations, it can help users accumulate professional word combinations and vocabulary. Ultimately, it improves users' reading skills.
Basic Persona
They use their free time to learn English.
Students
Female
Goal: To improve their English skills through the App.
Needs: Chinese and English bilingual switching, real-time, rich content, and free.
83.6%
64%
Learning needs
Users can correct translations and earn achievements. The increasing difficulty of earning badges motivates users to keep translating. The use of highly saturated colors and rounded corners in the badge design gives it a friendly, youthful feel.
Timely interactive feedback. Timely notification to users when their translations are liked, approved, and displayed
Encourage users to compete by showcasing other users' translations and their achievements.
To increase the user's sense of honor, the achievement medals earned and the translations completed are displayed after the user's name.
Social needs
Competitive desires
Self-representation
Understandability & Fluidity
Add an "editable" icon after each Chinese-translated paragraph; add an explanation of the new feature in each article to convey the usability of the new feature and reduce the cost of user understanding.
Prevent losses from misuse by saving users' drafts in real time and confirming them twice before submission.
Timely feedback of translation results. After the translation is submitted, the user's translation is displayed in real time and marked. Meanwhile, users can find the translation records in their personal center.
Click to translate paragraph
Translate
Translating
Confirmation of submitting translations
Displaying translated content
other pages
Project Outcome
The XiMei translation feature was the design team's first attempt at an MVP project. Without using programmer resources, I led the team to quickly validate whether the product's core features had business value, and to continue to improve the solution based on feedback results and data statistics.
Conclusion:
The data shows that the translation function is a success. The corpus data has grown exponentially and operational costs have decreased significantly, while the error correction corpus provides valuable data for subsequent machine learning.
Internal testing
After the release
Number of articles
170/week
1950+/week
Average length of app usage
2.46min
4.78min
397/4000
3000+
Number of participants
Team ability accumulation
Incentive methods
material motivation
cash bonus
Cash exchange
Material incentives are more suitable for stage operation activities (e.g. translation contests) and cannot form long-term encouragement. We can choose scarce resources as rewards, such as one-year subscription to certain foreign journals and other special rewards.
Score rewards are given after each completed task, and the scores can be exchanged for cash. This type of cash reward is good for long-term accumulation of rewards and cultivating user habits; however, this type of reward needs to be prevented from cheating and needs to be reasonably distributed when designing tasks, as well as limiting the time of redemption.
spiritual motivation
Achievement Point
Personal Badge
Measuring the degree to which the user has achieved a goal allows the user to understand their progress and performance and motivates the user to continue participation in the task or goal. In terms of display, each time a user earns an achievement value, the text of the incentive is displayed.
Personal Badge is to recognize and reward the user for completing a task or achieving a goal.A "Personal Badge" requires more formality and the opportunity for full display and exposure, thus increasing the user's sense of honor and self-esteem.
Validating the feasibility of business assumptions through teamwork
processes
participant
responsibility
Planning Business Assumptions
Designer,
Project Leader
Define business objectives
Develop business hypothesis
Designing solutions
Setting key targets
Implementing solutions
Collecting and analyzing data
Collecting feedback
Validate business objectives
Improvement solutions
Full-fledged implementation
Designer,
Project Leader
Digital Marketing
Designer, Project Leader, Engineer
implementing Business
Assumptions
Verification results
Publishing business plans in a real environment
Not meeting expectations, adjusting solutions
meet expectation