Saurabh Nanda Personal Blog

How we used technology to complete the Goa Government Covid Survey before time

18 Apr 2020

Along with volunteers from Vacation Labs, I was involved in managing technology, training, and support for the recently conducted state-wide Covid survey in Goa (on a completely pro-bono basis).

Over a period of 3 days, 6,700+ surveyors visited every single household in the state of Goa, and collected 440,000+ survey responses (one response for each household, not individual person). In fact by the end of the second day, team leaders were reporting close to 100% coverage. The third day was primarily used for mop-ups.

This massive government exercise finished before the allotted three days! From conceptualization till kick-off, we had less than 10 calendar days. This is my account of how we achieved this.

Table of Contents

Acknowledgements

This would have not been possible without the efforts of every single surveyor who sweated it out on the field.

This would not have been possible without the state/district administrations working long hours in mobilizing and coordinating this massive human machinery.

This would not have been possible without the efforts of the “master trainers” and members of ITG, who helped in scaling the training and on-ground support.

This would not have been possible without the voluntary efforts of my colleagues at Vacation Labs.

And finally, from my point of view, the unnamed IAS officer is who brought all this together. If it were not for him, I would not have had the opportunity to get involved, and for all I know, the government might still be struggling with mountains of paper…

How did this start?

I am made to believe that the genesis of this massive exercise was the so-called Bhilwara model. We knew about it 7–10 days before it was trending on Twitter. Strangely, when it was trending on Twitter, the only thing being talked about, was the “ruthless containment”. However, there was also a door-to-door survey conducted across 10–20 days throughout the district of Bhilwara, which the Government of Goa wanted to replicate.

That is where this idea started.

Coverage of “Bhilwara model” on Indian Express

Coverage of “Bhilwara model” on The Print

I do not think that Bhilwara used technology to rapidly conduct the survey and collate results. It was a paper-based survey, and I do not know how they collated this data from lakhs of sheets of paper.

I was called-upon for help by the aforementioned IAS officer, who was concerned that the Government was considering to replicate this survey using pen-and-paper. He wanted me to quickly put together some technology solution that would enable the Government to collect all survey responses digitally (so that data-analysis is much easier later).

What was this survey about?

The Department of Health (Govt of Goa), wanted to primarily collect data about:

  • people (or population clusters) with ILI symptoms (influenza-like illness)
  • people (or population clusters) in other high-risk categories (i.e. elderly people, existing chronic illnesses, recent travel history, etc)

After a bunch of iterations and navigating HiPPOs, the final set of approved survey questions was arrived at:

The final list of survey questions
The final list of survey questions

Gegraphical location of survey response

Apart from these questions, the survey app also collected the geographical location of the survey response:

  • Lat/long coordinates, which were recorded automatically by the mobile app.
  • Geographical zone in which the household was located, i.e. Taluka => Village (or City) => Waddo/Mohalla (or Municipal Ward), which was basically an explicit question in the survey

The Technology

Attempt #1 - Custom mobile website

There is a back story to this, which I will avoid, but I was under the impression that the Government didn’t have any technology budget, whatsoever, for this exercise. So, on a pro-bono basis, I started writing a mobile website as the technology backend to conduct this survey. I had barely written the login screen when I realized that idea was bound to fail if it didn’t have offline support.

After spending a couple of hours researching PWAs, (basically, offline webapps) I was convinced that treating this like a hackathon isn’t going to work. We needed an existing and well-tested mobile app to ensure that survey responses could be collected without network coverage as well.

Attempt #2 - AppSheet

A friend suggested AppSheet as a no-code/low-code solution to quickly build a mobile-app. I gave it a spin and found it to be a reasonable solution, and discussed its pricing with the unnamed IAS officer.

Till this point we were under the impression that about 200 surveyors would go out into the field. With a $5–10 per user price range, the total was coming to approx Rs 75,000–1,50,000 ; an amount that the aforementioned IAS officer was certain he would be able to accommodate within his budget (or however else he was planning to turn the cogs of the government machinery). We were nevertheless planning to reach out to AppSheet to get a reduction, if not a complete waiver, on the usage charges.

I had built out a PoC of the survey form using AppSheet and was scheduled to demo it in a high-level meeting the next morning, when a friend alerted me to this tweet:

From 200 surveyors, we had suddenly gone up to 7,000 surveyors! There’s no way the AppSheet pricing could now work.

(PS: Also, while testing, I noticed that the AppSheet mobile app froze randomly, and the offline mode didn’t work reliably).

Workable solution - Using Zoho Forms

I began searching frantically for a cheaper AppSheet alternative, and realized that we didn’t really need a low-code/no-code way to build a mobile-app. We just needed a survey application that:

  1. was cost-effective, and
  2. could work offline

I looked at a bunch of surveying products - Wufoo, SurveyMonkey, Forms.io, etc - and all of them were too expensive.

Then I remembered Zoho Forms.

We had recently moved to Zoho One at Vacation Labs and had spent a lot of time in automation of sales, finance, and support process on top of various Zoho products.

It wasn’t perfect, but it looked liked it could get the job done.

The Pilot Survey - Where trouble began…

Finally, the solution was green-lighted and the government machinery started cranking. Employee databases were being updated. Survey teams were being formed. Opposition was “slamming” the government. And PILs were being filed. The usual.

I urged the Government to do a pilot survey before rolling out this untested (for us!) solution across 7,000+ surveyors.

And the pilot survey was an eye opener!

  • The very first step of on-boarding 10 surveyors to the app turned out to be a fiasco. The only way to add users to Zoho Forms is to invite them via email. It turns out, not everyone has an email address, and even if they do, they don’t remember the username and password of their email accounts!
  • Even if they end-up getting the invitation link over email, they get stuck while setting their account password. You see, Zoho doesn’t like “abc123” as a password!
  • The Zoho Forms mobile app has two buttons to login. If you end-up clicking the second one (“Signin with Google”), it ends up creating a completely independent organization using that email. Once created, there is no way you can invite that email address as a user to your organization.

After somehow struggling with on-boarding 10 surveyors for the pilot, we finally began the actual survey on the field. The app worked perfectly:

  • All survey responses were recorded
  • Responses recorded without the network were synced as soon as the app connected to a network
  • Lat/long coordinates were collected for most responses, except where there was some problem with the GPS itself

However, there was no way we could on-board 7,000+ surveyors to Zoho Forms given their default user on-boarding processes. And so…

Finding solutions with Haskell, and what finally went live

That night, I put in an all-nighter working around the Zoho signup process by writing a script in Haskell. It was a big hack-job. But it got the work done! (Broadly, it delivered the Zoho Forms invitation email to a domain under my control, extracted the invitation link out of the email, and activated the account by mimicking user actions).

That Haskell script, over the course of next few days, grew to become the backbone of the entire system. Basically, Haskell filled in all the gaps that were missing in Zoho Forms for our use case:

  • Creating and activating users without requiring any action from the user itself. It was important for the username and password to be easy to communicate. Therefore we came up with something that could be derived from the surveyor’s phone number itself:

The username was p[phone-number]@covid.vacationlabs.com and the password was also based on the surveyor’s phone number itself.

  • Creating 8,000+ users based on data provided by various Deputy Collectors on Google Sheets (yes, we on-boarded more than the projected 7,000 surveyors! )

  • Creating and configuring 582 survey forms - approximately one for each village, or city, or part thereof. The village (or city) name, along with the sub-divisions (waddos/mohallas) were in a Google Sheet being kept up-to-date by all Deputy Collectors.

  • Sharing each of those 582 survey forms with the 2–6 member team that were covering that village. Once a surveyor signed into an app, they would see only the survey form relevant to them - not all all 582 of them!

  • Deactivating users in bulk - this was required at the end of the survey

  • Fetch number of survey responses at each village and waddo level and publish it to a public dashboard

Dashboard displaying aggregate stats that was used to monitor progress<br/>(written in Haskell)
Dashboard displaying aggregate stats that was used to monitor progress
(written in Haskell)
  • Finally, download raw survey responses from 582 different forms at the end of the survey. This was run only once, and that too, directly on a laptop provided by Department of Health Services.

The People

While technology was an enabler, at its core, this was a massive human effort. None of this would have been possible without the 7,000+ surveyors who sweated it out in the heat for three days going to every single household in Goa.

All of this was mobilized by the state/district administration, which consists of Collectors, Deputy Collectors, Mamlatdars, etc.

Formation of survey teams

I believe the plan was to have approx 1,700 teams of 2–4 people, each headed by a BLO (Booth Level Officer) of that area. BLOs are in-charge of a part of the electoral roll and know the local people and houses inside-out. The other members of the team were anganwadi workers, teachers, electricity meter readers, and other government officials.

Scaling up training and communication

Once the pilot survey was successful, and this had to be scaled-up, I reached out to my team at Vacation Labs with help in training and support. With the help of all those who volunteered (from Vacation Labs), and the district administrations, we trained close to 100+ “master trainers”, who then went on to train the 7,000+ surveyors.

Along with us, Dr. Utkarsh Betodkar, the State Epidemiologist of Goa, spoke to the “master trainers” about the core medical questions.

Training the master trainers

A message to the surveyors (video)

We realized that while the “master trainers” could do a decent job of helping surveyors with the mobile app itself, it was not right to expect them to deliver medical guidelines in a second-hand manner.

Therefore, with the help of Jump Cut - another Goa based company - we quickly recorded a video for the surveyors where Dr. Utkarsh Betodkar, the State Epidemiologist of Goa, explained the significance of the core medical questions to surveyors.

Scaling support on the three days of the survey

Irrespective of how much time we spent in training, we knew that there would be chaos when the survey actually began, and the team of volunteers from Vacation Labs would not be able to handle the volume of support issues.

So, we were shielded from the obvious/common support issues by a team of ITG members, and only L2 issues were escalated to the Vacation Labs team.

Comments / learning

Comments / learning about working alongside government machinery

Hardworking, when required: Most of us are accustomed to think of Govt administration as lazy and inefficient. My first hand experience was the exact opposite of this. Throughout the 7–10 days, I’ve had Deputy Collectors stay-up till 11pm in the night to provide us with the required data. I’ve called people up at midnight and found them to be up and working. And then found them in a 9am meeting the next day on-time, before me!

Better public communication was required: It would have been better, if the purpose of this survey, the exact questionnaire, and how the data will help in tackling the Covid pandemic, were shared with the citizens well in advance and over multiple communication channels (press conference, Twitter, Facebook, YouTube videos, etc). In my opinion, a few comments in press-releases were not enough, especially when one is expecting the public to share personal information with the Government.

Lack of structured geographical data: I was surprised to find out that no master database of geographical divisions and sub-divisions was readily available within the government machinery. Theoretically, https://lgdirectory.gov.in/ is available, but, best of luck trying to get a unified list out of there. Moreover, it doesn’t seem to go down to waddo/mohalla level, which is what was required for this exercise. We ended up working with a badly formatted Excel file as a starting point and then banking upon Deputy Collectors of all regions to get this data into shape.

Who is in charge? Who is doing what? The government machinery seems to have way too many (overlapping?) departments. As an outsider working alongside them, it was very hard to for me to figure out who was doing what with respect to this survey.

Email is the Achilles’ heel of the Government: Not one person was comfortable in operating their official @nic.in email ID. I was receiving official Government communication through free email accounts on Yahoo or GMail. Even when I sent emails on their “official Yahoo/GMail” account, most officials didn’t reply. I had to call-up the individual over phone, confirm that they received the email, and then be satisified with a verbal response, or a one-liner reply on WhatsApp!

Comments/learning about Zoho Forms

Core product works reasonably well: We had two core requirements from the surveying app - (a) ease of use, and (b) should work offline. Zoho Forms performed reasonably well on both these counts.

Waived off all charges: What’s more, Zoho Forms graciously agreed to waive off all charges for this exercise. As per their standard pricing, it would’ve cost the Government Rs 24,000 per month for the Zoho Forms platform.

Zoho Forms mobile app worked across all devices flawlessly (both Android and iOS). Every surveyor found the survey form easy to use and the app was extensively used in offline mode.

Having said that, no product is perfect. And no product fits all use-cases.

Because the Zoho Forms UI was not designed for our unique use-case, we faced the following “workflow” issues. However, the silver-lining was that the entire product is built on-top of internal JSON APIs (not public), which I discovered and used in the Haskell scripts:

  1. No bulk operations: We needed to create users in bulk1. We needed to create forms in bulk. We needed to share forms to various users in bulk. We needed to download reports in bulk. None of these features were available. Thankfully, that is where Haskell filled in the gaps and made rapid deployment possible.

  2. No analytics dashboard (not even basic numbers): Surprisingly, nowhere in the Zoho Forms UI does it show the number of responses for each form at a glance. We ended up writing our own dashboard by scraping the actual numbers buried deep within the UI.

There are four things that I feel are worth fixing in the Zoho Forms product (in fact, I reached out to Sriram, the Product Manager, and he’s already got these on his roadmap2):

  1. No validations in offline mode: Zoho Forms does not perform any validation on the survey form if it is submitted offline. For our use-case, this caused problems when the mobile number was not exactly 10 digits. The survey response failed to sync with the server when the phone came back on the network. This resulted in a non-intuitive error message and multiple escalations to L2 support.

  2. Issue with user on-boarding: It seems that the same email address cannot be invited to two Zoho Forms “organizations”. A number of L2 support escalations came because surveyors ended up using the “Sign in with Google” (or probably “Sign up”) link on the app, and creating a new organization for themselves!

  3. No ability to have user groups: Ideally we would’ve liked to add users to groups (based on their villages/cities) as we were on-boarding them. And then, share survey-forms with these user groups. None of this is possible in Zoho Forms currently.

  4. Weird permissions/roles: Strangely, when you add a user to your Zoho Forms account, they get a default role of “User” (which allows them to create new forms). There is no way to change the role of the user to a “Respondent” (which has lesser privileges) unless the user signs-in at least once.

Comments/learning about data privacy

Need better framework for data collection: The need of the hour is to have pre-prepared data policy frameworks which can be applied to the scenario at hand. For example, in the current scenario there are genuinely good use-cases for, both, raw data, as well as the anonymized/aggregate data:

  • Raw data: As mentioned earlier, the expected use-cases of the raw data should have been documented and share widely beforehand. In the current scenario, the raw data would be used by Department of Health Services for various activities directly related to handling the Covid pandemic. Example: for re-verifying symptoms with specific households, for prioritizing Covid testing, for recommending home quarantines for high-risk cases, etc.
  • Anonymized/aggregated data: Similarly, there should be a pre-prepared policy framework on how the anonymized and aggregated data would be shared with other departments (eg. for public policy planning, targeted delivery of services, planning development projects pertaining to certain age-groups)

Define opt-out provisions: While the Government was clear that the responding to the survey was voluntary, it would have been much better if a mechanism for an opt-out SMS would have been put in place, as well. After the survey, an SMS would have let the respondent know the actual response recorded on their behalf, and a way to delete it from the survey database within a stipulated time, if they chose to do so.

Formation of neutral committees as “data stewards”: With the government collecting more and more data digitally, it is becoming increasingly important to have neutral committees that have “stewardship” duties over this data. The following is all wishful thinking, but is the following not possible:

  • Such committees consist of relevant Govt administrators, ruling party AND opposition party members, and a significant number of members from civil society (including NGO and social activists, not just IT company CEOs).
  • Data-at-rest is encrypted, and can be decrypted only when a quorum of the committee members agree to decrypt it. (Is there a way to put a time-box around how long the data will remain decrypted? I don’t know!)
  • The committee decides and votes upon use of raw data, anonymized/aggregate data, and when the data should be deleted.

Comments/learning about Haskell

I plan to do a separate post sharing first-hand experiences of using various Haskell libraries in a high-pressure and time-sensitive environment (also where I was massively sleep deprived!)

Please tweet to me at @saurabhnanda if you’re interested in first hand critiques about any of the the following libraries: http-client, gogol, aeson, servant, lucid, tagsoup, cookie, concurrent-output, lens, lens-aeson, streamly, and token-bucket

Results and Impact

The actual results and impact remain to be seen over the next few weeks. However, I am told that the Department of Health Services has already started identifying high-risk clusters and will be reaching out with the next course of action (prioritized testing, in all probability).

I am no longer involved in any way with handling or analysis of the data, so this is just my best guess.

Important Caveats

Please note this is my personal account about the happenings before, during, and after the Goa Covid Survey. This is has been written with an intent of promoting transparency and sharing my learnings and recommendations for anyone planning to conduct a similar exercise in the future.

Footnotes

  1. There is something called Zoho Directory for centralised and bulk user management, which is ideal for large enterprises. Since Zoho is a large company with multiple independent product teams, Zoho Directory is still in the process of being integrated with various Zoho products. By the time I came to know that Zoho Forms had some sort of (unpublished) integration with Zoho Directory, I had already crossed that bridge. ↩

  2. Comments from Sriram (Product Manager at Zoho Forms):

    We have already started to work on some items, like: (a) basic validation in offline mode, (b) choosing a role when adding a user, (c) basic dashboards for forms, etc.

    There are some workflows which are possible with some workarounds, like bulk user addition, which is possible via Zoho Directory, but we are planning to add them directly into the Zoho Forms product. 

    We might not support some use-cases in the product at all, like bulk creation of forms.

    On the whole, barring the offline validation case, most other features mentioned above are for enterprise use, which are already being worked upon, but haven’t been released yet.

    ↩

Related posts