How Did I Start Working in the Disability Sector

Getting Initiated

In DA-IICT, we had introduced design thinking in the second semester of BTech(ICT) program. This was followed by an internship in summer(later changed to winter) in a rural area and a Zero Credit Design project in the second year. Idea was to let them explore some area of societal benefit for a full year without any pressure of grades and timeline.

Two students, Pallavi and Ramya(both from Hyderabad) approached me(most likely in 2007) to work with me on design project with an interest in helping persons with visual impairment. They visited Blind People’s Association in Ahmedabad to get an idea of various needs. Later in summer vacation, they came across news about a girl, Bhawana from Chennai, who had passed class 10th. Normally this does not make news but Bhawana was not in a position to move or talk. She did not have fine finger movement just a coarse movement of hands. In spite of these conditions, her passing of 10th was worthy of news. Pallavi and Ramya visited her in Chennai to understand how she studies and carries out other functions.

After visiting Bhawana, they met me at University and described to me her condition. Bhawana also had a device (probably developed in IIT Madras) that allowed her to operate television by changing channels or volume but the system could be configured only for one parameter to be controlled at a time. I suggested that let us work to develop a system that can allow Bhawana to control both volume/channel together by using her coarse hand movement. We could use up-down/movement of the hand to control the Channel and right-left movement to control volume. They worked on the project and could use an accelerometer to identify movements but did not go to the stage of connecting that to TV control. I realized the significance of this type of work and suggested this to the next batch of students Prashant and Mohit. They could complete the project and made a working prototype. We improved the prototype from a lab model to something that could be tried by users. This device, named, CePal(short form for Cerebral Palsy), won HP Innovate 2009 award at the national level and the team led by me had a paid trip to HP Labs in USA.

I approached National Trust (an autonomous body of the Ministry of Social Justice and Empowerment) for support of this project. Mr Atul Prasad(CEO) and Poonam Natarajan(Chairman – I had a chance meeting with her on a train journey) were very helpful in this regard. I was also using an IBM fund that I had received in 2004-05 with no specific project linked. A large number of students/research engineers worked on these projects. These are mentioned in the “The Team” section.

I found that very little work was being done in this area within India and there was a tremendous need to meet the unique requirements of each PwD and bring about changes in quality of life in a miraculous way. I kept persisting with this line of work and it led to many innovations based on the needs of Persons with Disability(PwD). It became an important part of my life and a mission. I had to reach out to users and spread across the country. Social media and word of mouth were very useful. I used to spend extra time whenever an occasion to travel to any part of the country was available. It required visiting houses in far-flung areas. Sometimes some NGO or other organizations would help in the matter and sometimes it would be all personal contacts. Interestingly everyone expected that I would also find funds to support any expenses and everything would be done free. For me, it became a life mission and the developments described below are an outcome of this and I continued to devote a significant amount of time to this area especially till 2013 before joining TIFAC. In fact, I received NCPEDP- Mphasis Universal Design Award given on Aug 14th 2012 on the same trip that personal interaction with TIFAC selection committee meeting took place. Just before this I had also received Bihar Gaurav Samman from Shri Narendra Modi , then CM of Gujarat. My work in disability sector was an important consideration for this as seen by the slide shown at the time of award and announcement made.

As mentioned, our initial development was to use gestures but while handing these systems out, we found many other users, who could not use this device since they did not have hand movements. To cater to such needs, we decided to go ahead with the development of similar systems based on any input that the user was able to provide such as voice, eye movements, eye blink, facial expressions, conscious thought or muscle movement etc. While some of this work remains to be completed, we have already been helping users with our systems and they have benefited a lot. We have been identifying the needs of various users around the country by making visits and giving talks to the user community as well as organizations involved in the disability sector.

In this write up we give technical details as well as how these systems have been affecting the life of various users and related possibilities. It is not possible to cover all users and there is a heart-touching story related to each user. It would require a full book to write about all the cases. Here we give a sample of few such users.

​Methodology

We have taken advantage of few technological developments in carrying out this type of work. We list them one by one:

​ IR based Remote Control

We used IR-based remote controls available today. Many of the electrical/electronics equipment come along with IR-based control. In addition, it is possible to easily add IR control to some of the equipment by commercially available cheap devices. While TV, Air-Conditioners, Music systems come with IR remote, it is possible to add such remote control facility to Light, Fan, Computer and any other electrically operable system. By sending the same IR control commands by any other means, we can get the equipment to respond as it would by its normal remote control.

​Sensors

Another technological advance has been in the area of “sensors”. Due to the availability of MEMS(Micro Electro Mechanical Systems) technology – the cost, size, and power consumption of sensors have gone down. Its integration into portable embedded devices has become much easier. Body part movements can be captured using accelerometers and gyro sensors. Similarly, pressure sensors available can be used to make cheap sip-n-puff devices based on air pressure created in a pipe by mouth.

​ Brain-computer Interface

Another interesting development in this has been the cheap and user-friendly “neuro-headset” for playing computer games. These can in turn be used as an EEG sensor using which one can capture brainwaves(EEG waves). With the wireless transfer of data and real-time analysis possible now and, it opens up a lot of possibilities.

​ Touch Sensor based Phone/Tablet

The availability of cheap touch sensor-based Phones/Tablet has brought a number of technological possibilities within reach of Indian users for the last 10 years or so. The touch screen acts as both an input and output device. A slight swipe or tap of the finger can be used as input. Built-in wireless communication, powerful processors, easy software development along with a variety of peripherals create a myriad of possibilities for persons with disabilities.

​Our development of Assistive Technology

We describe how we have taken advantage of such technological advances to help persons with disabilities in India. In the table below, we show the various technological development that we have carried out based on the different abilities of the persons and the status of such technology:

CePal : Hand gesture based Environment Control Unit (ECU)

As mentioned above, the first work in this regard was for persons with restricted finger movement (e.g. some with Cerebral Palsy) and allowed persons to control the environment around them(including operation of a computer) using hand gestures even if they could not press remote control buttons. The system further allowed the use of up to four different pieces of equipment with a single hand/wrist-mounted system. The early version of this system, “CePal”, won Hewlett Packard Innovate 2009 first award and was supported by National Trust for further development. This low-cost device has been given to users as per the requirements of National Trust in two different versions to suit individual needs : (1) RF-CePal (2) Mini-CePal. For all users, it gave them much-improved self-reliance as compared to their dependence on others.

Mini-CePal : This device is made in one piece and mounted on hand/wrist. It monitors the hand gestures of the user and sends Infra-red commands through an IR LED mounted at front of the box. While it has the capability to operate four different equipment, it has been found most suitable to operate a computer for those who do not have fine finger movement to press keyboard buttons or button-based remote control. 50 of these devices were distributed by National Trust to various user agencies.

RF-CePal : This system consists of two parts :(1) Hand device, (2) A fixed base station

Hand device monitors the hand movement using an accelerometer in two directions: Up/Down, Left/Right. It also detects taps by users. All this is transmitted to the base station by wireless communication using Zigbee protocol. This system supports up to four pieces of equipment. One can change the equipment being operated by tapping the hand two times within a defined interval. It also has 4 LEDs to indicate, which mode became operational. A fifth mode. “sleep mode”, is also defined, when the device ignores hand movements.

The base station acts as a gateway between the hand device and equipment to be controlled. It receives commands from the hand device using wireless communication. It interprets this information and prepared the control commands to be sent to the equipment to be controlled. It has IR LEDs mounted on flexi-tubes, which can point in the direction of equipment. Through this, it is able to control the equipment as per the intention of the user. This also has four LEDs, which light up to match with the LEDs on hand-device to indicate the equipment being controlled.

Ten of these devices were fabricated and handed over to National Trust. Some of the users are shown below. One user, Neha Sodhi, in Delhi asked her family members to leave her alone in the room so that she could watch TV as per her desire, whereas earlier she wanted one of them to be around to help her!

Now we describe our development of assistive technology based on Brain-Computer Interface (BCI) technology.

Systems Based on Brain-Computer Interface

In many cases of severe disability, the user is not able to move any body part and some are not able to speak as well. For such cases, we have used the latest development in the Brain-Computer Interface (BCI) technology to provide capabilities to users to be able to use computers and to control the environment(TV, Light, Fan etc) based on brainwave and head movement monitoring.

We have taken advantage of the development of neuro-headsets for playing computer games, where the brainwaves (EEG waves) of the user are monitored using these headsets to improve their gaming experience. With the games market being very large, these devices have become much cheaper and easier to use. This is making it possible to take advantage of this development to help persons with disabilities (PwD) at a much lower cost and for wider use.

We have been using the Emotiv Epoc headset, which collects EEG waves from 14 different locations on the skull as well as tracks head movement using a gyro-sensor. All this data is wirelessly transmitted to a computer, which can process this further. Depending on the user’s ability and need, we can recognize the facial expressions, emotions and train the system for recognizing certain conscious thoughts. This can be combined with the information from the gyro-sensor to provide a means to communicate in those cases, where users do not have any leg/hand movement or voice. Many users with spinal cord injury, Muscular Dystrophy, and other issues can immediately take advantage of this.

Brain-CePal: In addition to being able to communicate through a computer, we also developed a system by which users can control the environment around them. This is based on our earlier work of RF-CePal and we modified the system to make it operable through computer and control environment around them through neuro-headset. Depending on the user’s need, this can be completely done through user thoughts although as of now we have not come across any user needing this capability. This system can be further extended to allow persons to operate machinery through computers and improve their chances of employment. This system was demonstrated on July 1, 2011, as shown in the picture/video below:

Users of BCI based system:

We have carried out tests with several volunteers and demonstrated the working system on July 1, 2011. Since then we started trying to find users who could benefit from such development. Here are some of them, who are using the system or could successfully use it:

• Mr Suresh Karat (45 years Male, MBA, IIM Ahmedabad, 1993), Coimbatore – Brainstem Stroke left him quadriplegic with no speech since 1999 – His only way of communication was through eye blink, which was interpreted by his wife. On March 13, 2012 after a gap of 13 years, he started to type on computer all by himself using the system provided by us. He has written about his new ability here :

http://kesuresh.blogspot.in/2012/04/i-can-type.html

• Ms Charu Khandal(28 Years Female, Chief Animator, Ra.One movie, winner of film fare award), Mumbai : Her auto was hit by a drunk driver after a party she gave to celebrate award on March 25, 2012. She was very badly injured and right now she cannot control her hands/legs due to Spinal Cord Injury. Dr Abhishek Srivastava Head, PMR, Kokilaben Hospital, Mumbai requested me to let her try his system to see if she could use computer. After setting it up, she could start using computer very fast and went on to try painting. I promised her that she would be able to carry out her task of animation through the system and would see a much improved quality of life. Unfortunately we lost Charu in 2017. See my blog Remembering Charu Khandal

• Mr SS (48 years Male, Muscular Dystrophy, No control over leg/hand but good voice), Delhi – While he had voice but due to noise he found using voice recognition to type in computer very difficult. He preferred to use our BCI based system, which he started practicing since May 30, 2012. He wished to remain anonymous till he had enough practice with the system so that he can demonstrate it to other users as well as media so that many more can benefit. He wanted to be able to control TV/Light/Fan/AC and be able to open doors/gate and drive wheel chair though this system. I had hoped to provide them one by one to him when he is ready to use them but unfortunately we lost him few years ago.

• Mr Harishchandran (60+ years Male, Lost control over hand/leg movement due to stroke, very feeble voice), Mumbai: He was admitted in Kokilaben Hospital during my visit in 2012 and Dr Srivastava requested me to see if he could use computer. My first reaction was that he may not be interested in using computer but indicated that due to his interest in share market, he needs to use computer. We set up the system for him and for first 15-20 minutes he could not use the system at all as he found it difficult to focus. Then suddenly he improved dramatically and started to type on computer very well.

• Mr Prasad (45 years Male, School Principal, Spinal Cord injury due to Road Accident), Hyderabad : He has been paralyzed neck down since two and half years. He could start using computer in about 5-10 minutes. He has slight hand movement and is hoping that it would improve further. After his condition stabilizes, he would decide the best option for him to access computer.

This list is kept growing day by day but it is not possible to cover all of them here.

Touch Screen based computer access and communication system

As mentioned above recent technological development in mass-market products has brought many touchscreen-based systems to market like mobile phones/tablets etc. In particular, Android-based systems from multiple vendors has brought the price line so low that today for Indian economic condition, they have become very useful. Based on our interaction with various PwD and organizations serving them during my summer 2012 tour to many cities, we decided to take up the development of touchscreen-based cheap computer and communication system from regular use as well as teaching and learning point of view. We had already developed a couple of applications for this purpose during early 2012 and some of them were already tried by users and user organizations as well as family members have shown interest in distributing them to their users/wards. Some of the applications are shown below and their users given:

Speak-CePal – This has been developed to help those, who are non-verbal but have slight finger movement. This has been found to be the case of many Cerebral Palsy patients, Autism patients as well as many stroke patients of older age. It has a number of predefined icons (default 8), which can be extended to 16 or 32. Each icon is associated with a prerecorded sound. Initially, we had sounds recorded for English, Hindi, and Gujarati for both male and female voices. This would be extended slowly and more possible combinations of commonly used sentences would be added as well. The first version of this application was released on Aug 13th, 2012 to Polio Foundation, Ahmedabad, and some individual users like Rutvij(an Autistic child) and Devanshi(A dystonic child) as shown in the picture. This has also been tried by a number of users at Pearl Special Needs Foundation, Ahmedabad, and SPARSH Paediatric rehabilitation clinic, Ahmedabad.

A screenshot of the default setup is shown in the figure below. Touching any one of these speaks the corresponding sentence mapped to that icon in the language/gender of the selected option. Both the recorded voice and icons can be customized by the user. If a person does not have fine finger control, there is an option to let icons scroll one by one so that the user can touch any part of the screen to activate the voice.

We also developed an enhanced version of this to not only cover prerecorded voice but also carry out speech synthesis of typed sentences. Optimization was carried out to minimize the number of keystrokes needed to enter the sentences to save time and improve interactivity. A special system was prepared to let Madhuri (IICP, Kolkata) use the system through her head pointer as shown in the picture.

This system would be flexible to let Jeeja Ghosh, Dipak Ghosh and few other users at IICP, Kolkata use this to communicate and teach children. They are seeing a demo of this system through video conferencing and express their happiness over a major change in the quality of life this is going to bring to them.

In addition to the above-mentioned devices and users, I tried to understand the needs of different users and tried to develop solutions to meet their requirements. Sarita Dwivedi(shown below) lost both her hands and a leg in a childhood accident. I got a special electric wheelchair developed with help of Mr Bhargav Sundaram(Callidai Motor Works, Chennai) that could be operated by her feet.

Read about my interaction with Sarita Dwivedi in my blog Sarita’s dream came true!

Among many children that I met, Ganga (Shishur Sevay, Kolkata founded by Dr Michelle Harrison) is special to me. She was 10 years old, non-verbal, and had many other issues. The first time when I tried using neuroheadset with her, she simply would keep banging her head to the chair as if to get rid of it. I met her again after one year or so. By this time I had learned better about how to connect to children. This visit was more enjoyable for her and you can see Ganga playing music on a tablet. In the next picture, as I hold her hand, she seems to be communicating with her eyes.

Many NGOs were in touch with me. HI5 CDC, Mumbai run by Jasmeet Arora put me in touch with many users in Mumbai. During one of such visits to her center, we are letting Hitansh(A dystonic Cerebral Palsy patient) try our BCI-based system to type on a computer, as her mother watches with pride.

A boy in Mumbai is operating a light/fan using a tablet as her mother helps him out during one of my visits organized by Hi5 CDC.

Sanjeevani India, Delhi had organized a meet with users and parents in 2012 for a whole day. Here I met a lady with severe mental health issues. Read her anonymous story in my blog: Miracle recovery with my help!

As I kept meeting more and more people, my desire to help them in whatever way possible kept growing. Some of the things that I had tried on myself to help with my own health issues, were shared through my blogs and have helped hundreds of users, even though I do not come to know about most of them. Here are few examples of those too:

Liver/Gall bladder cleanse process based on Andreas Moritz Book

Hope for Psoriasis (and possibly other skins issues e.g. Vitiligo/Leucoderma, Eczema etc)?

सोरायसिस एवं विटिलिगो / ल्यूकोडर्मा, एक्जिमा इत्यादि चर्म रोगों के लिए आशा की किरण!

Miracle Recovery For A Young Parkinsonism Patient!

Based on these experiences, I am searching for ways to solve many chronic health problems affecting a very large segment of populations. A discussion on this can be seen in My thoughts on healthcare and wellness.

Media Coverage

Media has been covering my work for more than 10 years and it has helped me to reach out to a large number of persons. Few examples are here:

The Team

A number of DA-IICT students had participated in the early phase of the development of this device starting from Pallavi, Ramya, Prashant, Mohit and Hitesh. Subsequently, many research engineers contributed to make the systems more professional: Ishan Shah, Nancy Shah, Sainath Nambiar, Juhi Ranjan, Hiren Shah, and Ajar Roshania supported by Kiran and Firoza. Many organizations/individuals supported this work too e.g. National Trust, HI5 CDC, Mumbai, Sanjeevani India, Delhi, IICP, Kolkata, Family of Disabled, Delhi, Blind People Association, Ahmedabad, Sparsh Clinic, Ahmedabad, Polio Foundation, Ahmedabad, Pearl Special Needs Foundation, Ahmedabad and many more!

Concluding Remarks

Indian S&T research is dominated by the desire to publish more and more papers as our recognition system is dependent on counts. The impact of work should be more important than the number of papers. Work in the disability sector requires dedication to the cause and meeting each person’s unique requirements. Personally, for me, it is very satisfying to see smiles on the faces of people and I believe S&T is meant for that! Along with this journey, I made a large number of friends (and unfortunately many were lost to the world too along the way). Work in the disability sector has made me a much better human being and more positive as I have learned to

Count the ability and not the disability!