About STMT

Six Things Must TODO (STMT) is application for manage effective daily routine using Ivy Lee Method.

Group Member

• 한양대학교 정보시스템학과 김창회 changhoi0522@hanyang.ac.kr
• 한양대학교 정보시스템학과 김정은 wjddms04@hanyang.ac.kr
• 한양대학교 정보시스템학과 김은채 eunchai512@naver.com

Proposal

More and more people are complaining of difficulties in managing their time as the recent unprecedented situation of COVID-19 has limited outside activities and increased freedom of work. As work spaces and daily spaces become one, one is often interrupted by another.

Because human attraction and working memory are limited, it is virtually impossible to do multiple tasks at once. Therefore, doing a lot of multitasking means that self-interruptions and voluntary switching of tasks are often repeated. A study conducted in 2009 at the National Academy of Sciences of the United States shows that doing a lot of multitasking does not help the ability to switch tasks, but rather has a negative correlation.[*]

Ivy Lee method defines six things to do during working hours and forces the sequence of work to be done, making the routine a single task, not a multi-task. Our app makes it easy for people to apply and practice the Ivy Lee method. By taking advantage of this, people will be able to propose and practice ways to improve time efficiency beyond mannerism in their daily lives.

[*] Eyal Ophir, Clifford Nass, Anthony D. Wagner and Michael I. Posner (2009). Cognitive control in media multitaskers, Proceedings of the National Academy of Sciences of the United States of America, 106, 15583-15587.

Service Architecture

Mobile Application

1. Login
   Login screen is the first screen that users can see when they are not logged in to our application. User can login through their Google account. The button will lead the users to the Google login service, and our server will receive their information from Google. If the user is already logged in, the screen will show the main page immediately.

2. Main Screen
   Progress bar is on the top of the main screen. The colored part is the percentage of the tasks that is done by the user each day. Task lock and dashboard buttons are on the right and add button will be the only thing that users can click, if users did not set any tasks. Add button will disappear when users set six tasks, which is the maximum. They can make tasks by the add button and set specific details.

   • Task Detail
     When the user clicks the add button or the task itself, the screen will turn into specific tasks screen. Users can set the starting time, set the place where the task is going to be executed, and make detailed notes to each task. To-do list is also included in each task so that users can set a more detailed task.
   • Dashboard
     Dashboard shows the result of finishing the task in a bar graph. By clicking the graph or the day itself, they can select a date to view the results of the day, and the previous list of six tasks will be shown to the users. Previous tasks are viewed exactly like the tasks on the main screen.

3. Social
   In the social screen, there is a bar where users can search for their friend’s name and their task results, and an add button to add friends using google email. When users press the tab, they can take turns looking at the list of their friends or the list of other users who use this application, so the list looks different depending on the tab.

   • Search friends bar
     Users can search their friends by typing the name of their friends in the bar. Then the screen will show the list of the name that users have entered.
   • Add friends
     Add icon on the right side of the bar is a button that can add friends to the friends list. When the button is clicked, the screen will turn into Search user screen. User can enter friend’s google e-mail address to add to a friend list. However, if the user enters an e-mail address that does not exist, nothing comes up. Following and follower are shown through this screen.
   • List categories
     Tab is divided into two categories: Friends, All. Users can press friends tab to see the list of friends or press all tab to see the ranking of all the users who are using our application.
   • Friends list
     The list gives three information: rank, name, and percentage(score). Users can see the ranking by comparing themselves to the others. Percentage is for the ranking and is the sum of the percentage that users have completed the task. If there are users with the same score, they just go in order.

4. Settings
   There are four features in Settings Screen: Update user, Task related setting, Logout, and Clear cache.

   • Update User
     Users can modify their nickname by pressing Update user. In addition, they can set whether they want to expose their records or not. Users can make three choices: 1. Show their records to everyone. 2. Show their records only to their friends. 3. Don’t expose their records at all. Depending on the user, their records may or may not be seen by others.
   • Task Related Setting
     Task related setting can set the alert time and the task lock time. When the user set the alert time, the alarm will ring at that specific time every day. This gives users who have forgotten to work on their tasks a chance to remind them once more. If task lock time setting is set by the user and then the task is locked, the user cannot modify the task during the locked time, and the daily record will be stored based on the task lock time, which is used to score users in the list of social screens.
   • Logout and Clear cache
     Logout makes user log out the application. Clear cache does the same thing, but at the same time, clears application cache.

NUGU AI Speaker Scenario

User can give instructions to NUGU and the speaker will respond to the user by doing a specific action. Following are the instructions that users can give to the speaker:

1. Check ongoing task
   User can look up which task they are on and the name of it.
2. Complete the task
   Task will be completed when the user completes the task by the speaker.
3. Check the detailed task
   User can check the detailed task. NUGU responds to-do list inside the ongoing task to the user.
4. Check the remain tasks
   NUGU tells the user how many tasks are left and explains what number the next task is and what it is.

Every utterance command will be sent to the processing backend server through NUGU SDK. NUGU SDK processes the voice to string. And backend proxy server changes the string to valid command form and send it to the API server.

For checking the list of the tasks, NUGU answers back the task content as many tasks as user have left.

Methodology & Dataset

It used SKT’s NUGU AI speaker as a method of delivering commands.

Technical Overview

Voice recognition technology, combined with a variety of technologies, provides convenient services to customers and at the same time, expanding its scope by using customer response patterns.

Cloud-based voice recognition

Terminals such as mobile phones send data in a way of EPD (End Point Detector), voice capture and compression in streaming to the cloud server. Wake-up is with client. Since it must run for 24 hours and if wake-up is on the server, it will use our device’s data for 24 hours. Not only using the data we want but using all of them. Therefore, to send the data we want, wake-up should be in the client. Voice recognition service based on cloud sends the result to the device after the voice recognition is finished.

SKT carries its own voice recognition

Voice recognition learns DNN (Deep Neural Network) based on Acoustic Model and recognizes wFEST (weighted Finite State Transducer) base on Language Model. This method is the best for static. PLM (Personalized Language Model) is for dynamic and it can model personalized vocabulary to the place where it is going to be used before the voice input.

Online Decoder development using SIMD (Single Instruction Multiple Data)

By the fixed point optimization, computational optimization happened and the speed improved. Previously the respond time took 6 seconds and now it got faster to 0.7 seconds.

Sequence to Sequence Learning, CTC

Sequence to Sequence Learning applied n-best rescoring based on RNN-LSTM(Recurrent Neural Networks - Long Short-Term Memory models). Also extended RNN-LSTM to syllable based spacing models and applied CTC (Connectionist Temporal Classification) to pronunciation variants generator. Various DNN based technologies such as Wake-up, EPD, and speaker recognition are applied.

Development of 4-channel voice pretreatment

• AEC (Acoustic Echo Canceller)
  AEC gains voice recognition performance in a low SER (Standard Environmental Reference) environment. Also strengthen the long-tail control of acoustic transfer function by applying sub-block filter structure.
• RES (Residual Echo Suppressor)
  RES suppresses the residual echo that was not captured in the AEC.
• SSL (Sound Source Localizer)
  SSL predicts speaker direction information and enhances the responsiveness of wake-up towards utterance by applying noise and residual echo prediction.
• Beam-former & Post-filter
  Beam-former obtains voice without distortion by applying MVDR(Minimum Variance Distortionless Response) beamforming and has strengthen resolution by applying different filter constants by direction

Voice Recognition Architecture

AI Speaker put speakers and microphones as far away as possible because there was a problem with echo cancellation when they were close together. SKT puts servers on AI Cloud and provides machine learning technologies and a variety of services.

When the user speaks, the recording result from two microphones goes through an enhancement and the speech comes in through the audio recorder. First, distinguish whether wake-up is possible or not, then see if the signal is voice, divide the sound (by EPD), compress the part, transfer it to the server, and provide the user with the result.

Voice Recognition API

NUGU SDK

NUGU SDK supports Android, iOS, and Linux platforms. To simplify, it receives audio input through recorder, then use wake-up, and EPD to deliver voice recognition result. Voice recognition device in cloud service, and TTS is available to use.

Related works & tools

Efforts to manage time or organize to-do lists are old and countless apps and programs already exist. However, we have found that there are not many cases where specific management techniques have been applied, such as 6 things must to do, and in particular, the software used by actual users in the Ivy Lee method seems to be ‘6things’, ‘Ivy: Tasks & To-Do list, organize your life’.

6things is an iOS app with Ivy Lee method and features a simple and clear interface. Each day, the user organizes six things to do before and after bed and prioritizes them according to their importance. Although the app is very faithful to the Ivy Lee method, there were inconveniences such as not having a separate storage function and not being able to see the completed work again. There was also a problem that they could not provide time to look back on whether they had followed the Ivy Lee method well. On the other hand, our app not only stores past schedules on a daily basis, but also provides a dashboard of performance.

Ivy: Tasks & To-Do list, Organize your life (Ivy) are also mobile applications based on Ivy lee method. Unlike 6things, Ivy can see the completed items. The biggest feature of Ivy is that it can be divided into categories. Users can set priorities by category, and completed items are also sorted in a row by category. Categories may be convenient in that they can organize their work according to the situation, but in the end it goes against the Ivy Lee

Conclusion

• All the key parts of Ivy Lee Method have been implemented.
• It would be better if we designed it to share more functions with NUGU speakers.

Presentation & Demo