Artificial Intelligence Passes U.S. Medical Licensing Exam

ChatGPT

By Staff Reporters

***

***

Two papers show that large language models, including ChatGPT, can pass the USMLE. The papers highlighted different approaches to using large language models to take the USMLE, which is comprised of three exams: Step 1, Step 2 CK, and Step 3. ChatGPT is an artificial intelligence (AI) search tool that mimics long-form writing based on prompts from human users. It was developed by OpenAI, and became popular after several social media posts showed potential uses for the tool in clinical practice, often with mixed results.

According to Victor Tseng, MD, of Ansible Health in Mountain View, California, and colleagues, the results showed “new and surprising evidence” that this AI tool was up to the challenge. Tseng and team noted that ChatGPT was able to perform at >50% accuracy across all of the exams, and even achieved 60% in most of their analyses. While the USMLE passing threshold does vary between years, the authors said that passing is approximately 60% most years.

Source: Michael DePeau-Wilson, Medpage Today [1/19/23]

RELATED: https://medicalexecutivepost.com/2013/06/21/will-future-doctors-need-a-medical-license/

***

***

ORDER: https://www.amazon.com/Dictionary-Health-Information-Technology-Security/dp/0826149952/ref=sr_1_5?ie=UTF8&s=books&qid=1254413315&sr=1-5

COMMENTS APPRECIATED

Thank You

***

Cyber-Security Considerations for “Mission-Critical” Medical Devices

Join Our Mailing List 

Understanding the balance between new regulations (almost none) and guidance (in the form of non-binding recommendations)

By Shahid N. Shah MS

Shahid N. ShahTHEN …

In 2013, the Food and Drug Administration (FDA) issued its first cybersecurity safety communication, followed in 2014 by final guidance.

It struck a reasonable balance between new regulations (almost none) and guidance (in the form of non-binding recommendations).

NOW …

In 2015, the Federal Trade Commission (FTC) released a staff report entitled Internet of Things: Privacy & Security in a Connected World, in which it recommend that Internet of Things (IoT) style devices, which of course include medical and clinical devices, need to maintain a good security posture. It’s worth noting that the FDA, FTC, and other government regulators are centering on a few key guidelines.

Six Recommendations

The following six recommendations come directly from the FTC report:

  1. Companies should build security into their devices at the outset, rather than as an afterthought. As part of the security by design process, companies should consider:
  • Conducting a privacy or security risk assessment
  • Minimizing the data they collect and retain
  • Testing their security measures before launching their products
  1. Companies should train all employees about good security, and ensure that security issues are addressed at the appropriate level of responsibility within the organization
  2. Companies should retain service providers that are capable of maintaining reasonable security and provide reasonable oversight for these service providers.
  3. When companies identify significant risks within their systems, they should implement a defense-in-depth approach, in which they consider implementing security measures at several levels.
  4. Companies should consider implementing reasonable access control measures to limit the ability of an unauthorized person to access a consumer’s device, data, or even the consumer’s network.
  5. Companies should continue to monitor products throughout the life cycle and, to the extent feasible, patch known vulnerabilities

The FTC report and FDA guidelines are remarkably consistent. When thinking of cybersecurity and data privacy, engineers tend to think about authentication, authorization, and encryption. Those are the relatively easy topics.

*** circuit***

Mission Critical Medical Devices

For “mission-critical” medical safety devices, however, things are much more difficult and need to encompass a larger surface of questions, including but not limited to:

  • Asset Inventory: Is the device discoverable, and can it associate itself with standard IT inventory systems so that revision management, software updates, and monitoring can be automated?
  • Cyber Insurance: Does the device have enough security documentation to allow it to be insured by standard cyber insurance riders?
  • Patching: How is the firmware, operating system (OS), or application going to be patched by IT staff within hospitals (or the home for remote devices)?
  • Internal Threats: Has the device been designed to circumvent insider (hospital staff, network participants, etc.) threats?
  • External Threats: Has the device been designed to lock down the device from external threats?
  • Embedded OS Security: Is the device sufficiently hardened at the operating system level, such that no extraneous software components, which increase the attack surface, are present?
  • Firmware and Hardware Security: Are the firmware and hardware components sourced from reputable suppliers and free of state-sponsored spying?
  • Application Security: Is the Microsoft Security Development Lifecycle (SDL) or similar software security assurance process integrated into the engineering process?
  • Network Security: Have all network protocols not in use by the device been turned off so that they are not broadcasting?
  • Data Privacy: What data segmentation, logging, and auditing is being done to ensure appropriate data privacy?
  • HIPAA Compliance: Have proper steps been followed to ensure Health Insurance Portability and Accountability Act (HIPAA) compliance?
  • FISMA Compliance: If you’re selling to the federal government, have proper steps, such as use of Federal Information Processing Standard (FIPS) certified encryption, been followed to ensure Federal Information Security Management Act (FISMA) compliance?
  • Data Loss Prevention (DLP): Is there monitoring in place to ensure data leakage outside of the device doesn’t occur?
  • Vulnerabilities: Have common vulnerabilities such as the Open Web Application Security Project (OWASP) Top 10 been reviewed?
  • Data Sharing: Are proper data sharing agreements in place to allow sharing of data across devices and networks?
  • Password Management: Are passwords hardcoded into the device or made configurable?
  • Configuration Protection: Are configuration files properly check-summed and protected against malicious changes?

Conclusion

Your thoughts and comments on this ME-P are appreciated. Feel free to review our top-left column, and top-right sidebar materials, links, URLs and related websites, too. Then, subscribe to the ME-P. It is fast, free and secure.

ABOUT

Mr. Shahid N. Shah is an internationally recognized healthcare thought-leader across the Internet. He is a consultant to various federal agencies on technology matters and winner of Federal Computer Week’s coveted “Fed 100″ Award, in 2009. Over a twenty year career, he built multiple clinical solutions and helped design-deploy an electronic health record solution for the American Red Cross and two web-based eMRs used by hundreds of physicians with many large groupware and collaboration sites. As ex-CTO for a billion dollar division of CardinalHealth, he helped design advanced clinical interfaces for medical devices and hospitals. Mr. Shah is senior technology strategy advisor to NIH’s SBIR/STTR program helping small businesses commercialize healthcare applications. He runs four successful blogs: At http://shahid.shah.org he writes about architecture issues; at http://www.healthcareguy.com he provides valuable insights on applying technology in health care; at http://www.federalarchitect.com he advises senior federal technologists; and at http://www.hitsphere.com he gives a glimpse of HIT as an aggregator. Mr. Shah is a Microsoft MVP (Solutions Architect) Award Winner for 2007, and a Microsoft MVP (Solutions Architect) Award Winner for 2006. He also served as a HIMSS Enterprise IT Committee Member. Mr. Shah received a BS in computer science from the Pennsylvania State University and MS in Technology Management from the University of Maryland. 

Speaker: If you need a moderator or speaker for an upcoming event, Dr. David E. Marcinko; MBA – Publisher-in-Chief of the Medical Executive-Post – is available for seminar or speaking engagements. Contact: MarcinkoAdvisors@msn.com

OUR OTHER PRINT BOOKS AND RELATED INFORMATION SOURCES:

Product Details

Product DetailsProduct DetailsProduct Details

%d bloggers like this: