ICU Communication Group

Project Title/Device Name: TECHnically Speaking

Group Members’ and Primary Roles:

Mark Timothy Arroz:  Research
Adisa Narula:  Backend development
Ifetayo Estrada: Contact with “client/users”, Research
Will Hsu:  Front-end design
Yan Zhao:  Front-end design
Jason Beck:  Back-end development

Primary Population For Whom the Device is Being Developed:

The target populations for this project are the patients in the Intensive Care Unit (ICU) and the Pediatric Intensive Care Unit (PICU) who are mechanically ventilated thus having speech limitations, have limited use of their extremities especially the hands, and with some level of cognition which were brought about by their disability. This project aims to accommodate both the adult and children population.

Purpose of the Device:

This application allows the patient to communicate his or her various needs with the healthcare providers in the ICU. Moreover, this would also allow the patients to gain control of his immediate environment while in the ICU such as turning the lights on/off, turning the television on/off, etc. The edge of this device that we’re developing compared to the already established Alternative and Augmentative Communication (AAC) devices is the type of accessibility it would provide to people who have very limited or are unable to use their hands to navigate the application. The type of switch that can be triggered through head movements was taken into consideration; however, in this phase of the project, our focus is on building the application and the switch may come in the next phase.

Our main focus is to provide the patient a meaningful stay in the ICU which includes comfort, and to lessen the frustration they have given their current status. As much as possible, we want them to experience a certain level of independence while in the ICU.

Literature Review:

Current Practice in the ICU and PICU

Communication impairments in the intensive care unit is distressing for both adult and pediatric patients who have limited or no ability to communicate secondary to surgery, critical illness and or mechanical ventilation.  Being unable to communicate can be emotional, delay intervention, increase length of stay, and present challenges communicating end of life care. Where possible, nonverbal behaviors are the primary mode of communication for the critically-ill or nonspeaking patients, and these tend to be more ineffective, fatiguing and frustrating for the patient and clinicians. Asking simple questions such as yes and no could also be very limiting because majority of the input would come from the therapist or the healthcare provider (Khalaia et al., 2011).  Augmentative and alternative communication (AAC) tools have been explored as a means of communication to transmit messages (Costello, 2000).  According to Happ et al (2015), at least half of the patients who are on mechanical ventilators could benefit from an assistive communication device.

The use of low technology communication systems such as paper and pencil, picture boards, and emergency push buttons located at arm’s reach increased the patients’ abilities to express their basic needs if they have the ability to see, read, move their hands and/or write, etc. Advanced technologic devices such as Magic Slates, Alphabet boards, Eye Gaze systems are also currently being used in practice which actually requires lesser effort than the former methods.  There are challenges in current practice using both low technology systems and more complex AAC systems in that they aren’t accessible to patients that have very restricted movements especially the hands.

The Challenge:

                Our challenge is to develop a universal communication system for a tablet that can be used by the pediatric population and adult population in the intensive care units. We were also challenged to create a system that can be used with patients who have limited mobility of their hands due to the following conditions: Guillain-Barre Syndrome (GBS), spinal cord injury (SCI) and cognitive dysfunctions as seen in patients with stroke and traumatic brain injury (TBI). Patients in the neurological ICU have the least proportion of meeting the minimum standards for using AAC due to neurological insults that result in coma and altered consciousness (Happ et al., 2015).  Thus, it posed a challenge for us to develop an AAC device that could be universally used by patients with limited upper extremity movement or have limited cognitive capacities to communicate with clinicians and family, and control their environment.

Future Considerations in the ICU:

A study by Costello, Patak, and Pritchard (2010) has identified three phases for providing communication access for pediatrics in the ICU:

I) becoming aware of the environment

II) increased wakefulness and beginning simple communication

III) Need for a diverse communication access.

This project would most likely target the patients categorized in Phase I. They have also mentioned that environmental control switches are being used in some facilities; however, we have limited access to literature regarding this technology.

Grossbach, Stranberg and Chlan (2010) has emphasized different communication strategies when working with patients in the ICU:

(1) Establish a communication-friendly environment

(2) Assess functional skills that affect communication

(3) Anticipate patients’ needs

(4) Facilitate lip reading

(5) Use alternative and augmentative communication devices

(6) Educate the patient, the patient’s family, and staff about communication strategies.

We would strongly suggest that this type of categorization for patients in the ICU as well as the communication strategies be implemented to help the caregivers easily delineate the specific needs per patient.


NYUICUPICU Frankenstein Prototype - October 11, 2016
NYUICUPICU Frankenstein Prototype – October 11, 2016
This was used to control the lamp which was connected to a power switch tail.
Web Switch Button – This was used to control the lamp which was connected to a power switch tail. – October 23, 2016
The lamp was plugged into a power switch tail connected to a raspberry pie.
Output – The lamp was plugged into a power switch tail connected to a raspberry pie. – October 23, 2016










October 30, 2016 (Sunday) – We consolidated the feedback we got from the midterms presentation into our project. We’re finalizing the buttons we are going to place per page and how to maximize them. Addition of the body features and pain scale (Wong and Baker) was considered, and changing the “Emergency Call” button to “Call Nurse” button was decided. We still have to meet Tami to get further suggestions on how to make this app efficient and what features would be of top priority.

November 1, 2016 (Tuesday) – We divided ourselves into 3 groups. Some are working on making icons accessible. We used the icons from to make the icons we are using universal. The subsections of the major categories are also being finalized in order to make them easily accessible in a way that the patient would be able to easily locate them when looking at the icons. We are also deciding what libraries and languages to use to code the application and integrate the user interface design. For the week, we are planning to integrate the related literature into a more cohesive one.

November 16, 2016 (Wednesday) – We met with Tami in person and we’re discussing and finalizing on the features of our app. The pain category was refined. Addition of a feature where a patient can chat using a keyboard within the application is considered. We’ll try to do a visual scene display for the “I need…” category which can be configured in the settings to be used. The provider can switch to either icons or the scene display depending on the patient’s preference. Translation tool for icons with audio feedback is also considered a priority.

December 23-28, 2016 (Monday) – Consolidation of feedback. Added translation tool, visual scene display wherein the camera can be accessed to take a photo of the immediate environment to locate the patients needs. Addition of social media access to Facebook, Twitter and Skype.

User testing: 

We tested TECHnically Speaking with user Tami A. –  Speech Language Pathologist working in the intensive care unit.

Some questions we had going into the testing:

  • What do the call bell and lights look like in the ICU?
  • What short phrases should be quickly accessed in the menu bar?
  • How detailed should the pain scale be?
  • How do we present the different areas of the patient’s body when they find them painful?
  • How do we cover all the physical needs of the patient?

The results include:

  • access to social media including facebook, twitter, skype or video chat
  • visual scene display – option to take a picture of the room and toggle/scan through a grid to make needs known
  • simplify pain options – used Wong-Baker Scale
  • toilet button in the “I Need” menu
  • listing different types of suctioning
  • include “maybe” and “ I don’t know on the navigation menu
  • environmental controls to include Bed, TV, Light, and electronic window shades
  • options for translations to various language
  • male/female and adult/kids voice options

*As of the moment, user testing was limited to the SLP in charge because access to the patients in the ICU is difficult and the teaching phase for them to understand the app would also take time.

Design Process & Changes:

ICUPICU Version 1.0, Midterm Version – October 25, 2016
ICUPICU Version 2.0, revised version after midterms and in-class feedback – November 15, 2016
TECHnically Speaking, Final Version, version after meeting with the Tami
TECHnically Speaking, Final Version, Pain section
  • Finalized need categorization
  • Simplification of pain scale (Wong-Baker)
  • toilet button should be placed in the navigation menu
  • include “maybe” and “I don’t know” on the navigation menu
  • listing different types of suctioning
  • options for translations to various language
  • visual scene display – option to take a picture of the room and toggle/scan through a grid to make needs known

Final Design:

December 6, 2016 – This is the final design of our application. We included a switch that can be used to turn on the light. Other needs were also finalized. This photo was taken at the Prototyping Fund Showcase at the Tandon Maker Space


Generally trying to make this app as accessible as possible considering patients with the lowest function in the ICU posed a big challenge.Initially, we’ve spent a while thinking of the best switch to use for these patients however, we were told not to focus on the switch, but rather on the app itself. Some problems we had was how to split the initial categories of needs we’ve come up with into major categories and the patient needs to be included. The pain scale was also an item of concern as how patients would be able to express the kinds of pain they’re experiencing at the moment. This was resolved through a meeting with the NYU Speech Therapist who has more familiarity with the patients and their conditions as well as their general needs, and she was able to help us delineate which needs are necessary.


Materials obtained from ADA Fruit:

Raspberry pie (1)- $35

Wi-Fi modules (3) – $15 x 3 = $45 

infrared LED (1) – $1


Future Plans:

Translation of language to accommodate foreign-speaking patients to provide easier communication

Voice output modification that is to be able to change the speech output  into a male, female or kid’s voice.

The refinement of the visual scene display is another concern – preferably a more dynamic one and the ability to locate objects in the room.

To allow the app to control the recline features of the hospital bed as well as the window shades provided they are electronically controlled. 

How to send notification on a staff’s phone/computer when the patient triggers the “Call Nurse” button from the application.



Blackstone, S. W., & Pressman, H. (2015). Patient communication in health care settings: New opportunities for Augmentative and alternative communication. Augmentative and Alternative Communication, 32(1), 69–79. doi:10.3109/07434618.2015.1125947

Costello, J. (2000). AAC intervention in the intensive care unit: The Children’s Hospital Boston model. Augmentative and alternative communication, 16(3),137-153. doi: 10.1080/0743461001233


Costello, J. M., Patak, L., & Pritchard, J. (2010). Communication vulnerable patients in the pediatric ICU: Enhancing care through augmentative and alternative communication. Journal of Pediatric Rehabilitation Medicine: An Inderdisciplinary Approach, 3(4). doi: 10.3233/PRM-2010-0140.

Grossbach, I., Stranberg,S., & Chlan, L. (2010). Promoting effective communication for patients
receiving mechanical ventilation. Critical Care Nurse, 31(3),46–60. doi:10.4037/ccn2010728.

Happ, M. B., Seaman, J. B., Nilsen, M. L., Sciulli, A., Tate, J. A., Saul, M., & Barnato, A. E. (2015). The number of mechanically ventilated ICU patients meeting communication criteria. Heart & Lung: The Journal of Acute and Critical Care, 44(1), 45-49. doi: 10.1016/j.hrtlng.2014.08.010.

Khalaila, R., Zbidat, W., Anwar, K., Bayya, A., Linton, D., & Sviri, S. (2011) Communication Difficulties and Psychoemotional Distress in Patients Receiving Mechanical Ventilation. American Association of Critical-Care Nurses, 20 (6). doi: 10.4037/ajcc2011989.

Otuzoglu, M. & Karahan,A. (2014) Determining the Effectiveness of illustrated communication material
for communication with intubated patients at an intensive care unit.  International Journal of Nursing Practice, 20, 490-498. doi: 10.1111/ijn.12190.

Patak, L., Wilson-Stronks, A., Costello, J., Kleinpell, R. M., Henneman, E. A., Person, C., & Happ, M.
B. (2009). Improving Patient-Provider Communication: A Call to Action. The Journal of Nursing Administration, 39(9), 372–376. doi: 10.1097/NNA.0b013e3181b414ca.

Rodriguez, C. S., Rowe, M., Thomas, L., Shuster, J., Koeppel, B., & Cairns, P. (2016). Enhancing the communication of suddenly speechless critical care patients. American
Journal of Critical Care,25(3), e40–e47. doi:10.4037/ajcc2016217.