Project Name
Gemma Scan: A household dermatology device powered by MedGemma.
The Team
Evan Harbaugh: Data Scientist & Designer
• Concept development; software, hardware, and application design.
Problem Statement
I. Background
When I noticed an irregular skin growth on my back in July of 2024, I couldn’t decide whether or not to go to the doctor. It seems obvious that going to the doctor is the right decision, but I don’t think I’m alone that I’d rather not pay for a doctor bill for something that isn’t serious. However, it continued to change color and show warning signs of skin cancer so I decided to seek treatment.
When I did make an appointment with my in-network provider, I had to wait over a month to be seen. Thankfully my physician eased my worries when he said it wasn’t serious, but for over four weeks leading up to the appointment I felt helpless about the situation.
II. Problem
Jump to today, and there still isn’t a clinically reliable solution to bridge this gap. It’s up to individuals to decide that a skin growth merits a trip to the doctor, and even when an appointment is made, patients may have to wait months for feedback[1]. The five-year survival rate of melanoma decreases from over 99% at localized stages, to 35% at distant stages; highlighting the importance of early diagnosis [2]. Additionally, melanoma of the skin is the 6th most common cancer in North America, and the American Cancer Society estimates that 8,510 people will die from melanoma in the United States this year [3].
However, we may be able to improve those statistics with today’s technology. We can leverage tools like MedGemma with Google’s hardware and infrastructure to revolutionize dermatology with:
- A fast, accurate diagnosis of skin conditions.
- Support to monitor evolution of growths.
- Support to up-channel and expedite flagged conditions to dermatologists.
- A standardized platform to collect clean imaging data to provide additional training for AI models to achieve clinical accuracy.
Overall Solution
I propose an application that uses MedGemma to its fullest potential by delivering the power of computer vision to human hands. Mathematically, I do not believe it is possible to achieve 100% visual accuracy without standardized visual data to train any given convolutional neural network. In order to have the highest operational accuracy, I believe it is necessary to construct the same platform for both training and consumer-facing devices.
This solution remedies an existing need to evaluate household skin abnormalities with the medical use of AI, and opens up the visual platform necessary to train models at clinical standards. Other solutions would likely be less effective, yet Google’s data infrastructure and consumer electronics hardware allow for the perfect environment for such data-driven devices to meaningfully grow.
The core function of this device is called a dermatoscope (der-MA-toh-skope). Similar devices to the Gemma Scan exist, but most are designed and priced for dermatologist use. The only consumer-facing product I could find to purchase that compares to the Gemma Scan is the MoleMax PhotoMAX PRO Kit, currently priced at $4,056 [4]. However, as demonstrated in the video, a streamlined affordably-priced device is easily achievable via manufacturing advantages with circuit boards and camera sensors; especially with the optional support of Google’s fast and reliable GPU cloud capabilities.
Technical Details
I. Data & Devices
I used an Android tablet with a Raspberry Pi for the user hardware and connected the Pi via SSH to my local PC to run MedGemma 1.5 4B on 16GB VRAM and no internet connection. Although the prototype only serves as a proof of concept, it shows that the technical solution is clearly feasible. Additionally, the final product could be beautifully designed and cost-effectively deployed to the general public.
II. Prototype Design
The form factor was based on a Dexcom G6 Sensor Applicator, which originally housed a spring-loaded needle device. However, with the original contents removed, the applicator contained a manageable amount of payload space with sturdy mounting points for the new hardware. Prototype testing for the model mainly involved prompt engineering. Instead of fine-tuning the weights, I structured MedGemma’s prompts to adopt a new scale of 1 to 10 to rate the severity of skin conditions. Using a severity rating of 8/10 as the threshold for a skin cancer identification dataset, the model had a validation accuracy of 62% on a test set of 50 images, with a false negative rate of 48%. However, when the severity threshold was reduced to 7/10, the model had a validation accuracy of 70% on a test set of 50 images, with a false negative rate of only 8%.
The user-facing application stack builds on the fundamental functions of a handheld camera and ergonomic button. It then includes indicator lights that identify the status of the device and the software. The application connects a physical GUI to the device, and utilizes the touch screen. After the scan, the GUI returns indicator lights, a severity score, and MedGemma’s text analysis. The software also manages the data storage, and provides a visual interface for the user to see and access their previous scans.
Deployment challenges:
- It’s tough to tell if the skin condition is correctly aligned to the camera.
a. Fix: Add/connect a digital display to display image. - Raspberry Pi camera modules need at least ~5cm distance from subject to focus.
a. Use a larger USB camera to get optimal photo distances and dimensions.
In practice, many people would already benefit from an accessible household skin-scanning device, even without clinical accuracy. The software creates a rich database for the patient’s images, which provides context for a condition’s evolution over time. The device could be the deciding factor in someone deciding that a doctor appointment is necessary, and downstream data collected from all devices could enable the creation of one of the largest and highest quality medical imaging datasets imaginable.
Further evidence for the product’s practicality can be found in a study from 2019, which found that “machine-learning classifiers outperformed human experts in the diagnosis of pigmented skin lesions and should have a more important role in clinical practice”[5]. Yet, the same study notes that, “a possible limitation of these algorithms is their decreased performance for out-of-distribution images, which should be addressed in future research”[5]. Using a standardized visual platform like Gemma Scan could enable training to help close the gap for both the hardware and software causes that lead to the decrease in performance.
References
[1] Jairath, Neil K et al. “Validation of the Dermatologic Complexity Score for Dermatologic Triage.” Diagnostics (Basel, Switzerland) vol. 15,21 2765. 31 Oct. 2025, doi:10.3390/diagnostics15212765
[2] American Cancer Society, Survival Rates for Melanoma Skin Cancer
[3] American Cancer Society, Cancer Statistics Center
[4] MoleMax Systems, PhotoMAX PRO Kit
[5] Tschandl, Philipp, et al. “Comparison of the accuracy of human readers versus machine-learning algorithms for pigmented skin lesion classification: an open, web-based, international, diagnostic study.” The Lancet Oncology, vol. 20, no. 7, 2019, pp. 938-947. https://doi.org/10.1016/S1470-2045(19)30333-X.