Internal Meeting Notes

Date	Agenda	Notes	Action items

Date	Agenda	Notes	Action items
9/12/24	Check In Call	Foundational Paper Eboni wrote Discussion & Limitations sections, Joy currently reviewing (separate document, will send to team after meeting) Room for more results, may need to verify numbers Jen added data limitations References currently as comments, Ray can work on it or Pedro will complete it when back Results - see if there are any other areas aside from table of shortfalls and accomplishments table Direct messaging is not enough, need additional action Personalized interventions and uptake for providers Reorganizing table, showing success vs challenges by HC In methods - don’t need to say how many partners engaged, clearer in results Vaccine Hesitancy Haven’t done any analysis yet MPH paper focused on big push, mentioned list of narratives; not worried Previously identified themes? Deductive using identified themes? Inductive, grounded theory to verify & compare themes found from AI? Hybrid approach - traditional thematic analysis and include additional themes from AI analysis?
8/24/24	Check In	Foundational Paper Discussion has bullets, needs more elaboration Potential Journals - will want to select and then format paper to specific journal guidelines Journal of Community Health - peer reviewed, not clinical; may want to choose if not focused as much on clinical aspects depending on findings [First Choice for Foundational] Joy & Pedro have login Open access cost $5k BMC Public Health - Published MH Family and Community Health - Published MH HIPS paper [Second Choice for Foundational] https://edmgr.ovid.com/fch/accounts/ifauth.htm $2,500 for open access Preventive Medicine Reports - “softer” side of Preventive Medicine, may work with focus on intervention without data Vaccine - open access [Interesting for Vaccine Hesitancy paper] PLOS - Around $3k for open access Vaccine Hesitancy No updates yet	Eboni will take a look at Discussion section and add suggestions, write draft with Joy @Pedro Carneiro (Deactivated) will lead paper submission Will need to be submitted for CDC review before submission
7/25		Foundational Paper Intro, Methodology (could use minor additions) Stuck on Results, came up with outline Discussion - Overall findings from interventions, finding related to data Vaccine Hesitancy Sarah working on organizing transcripts on Confluence, page will be shared with ATSU team and will divide up work to analyze Next meeting Foundational paper - quick meeting second week of August - Thurs 8/8 at 2 pm 9/12 at 2 pm	@Sarah Price to fill in details on why partners dropped Eboni will work with Joy and work on Discussion section ATSU team will work on journal selection recommendations
6/27 Vaccine Hesitancy		Emory Student Thesis No conflict Did not include tables, people won’t be able to find her thesis likely Can cite her thesis Different methodology to analyzing data, can compare Include and emphasize co design of sessions Ray’s Analysis Amazon Bedrock - foundation is the transcripts Titan - can ask questions about themes and it cites which text transcripts documents it is pulling examples from Difficulty - program doesn’t know context High level identification of what should be quoted Separate into organization, conduct analysis on each group → put into table Not trying to replace grounded theory, only supplement it RAG - prevent large language models from hallucinating Next steps PHIT can send data to ATSU team for review July 25 meeting can cover both papers	@Raymonde Uy will separate transcripts into organization and run analysis, put into table
5/15 Vaccine Hesitancy & Foundational	Vaccine Hesitancy Updates Foundational Paper Next Steps	SharePoint issues, will resend link to ATSU team and may need to contact IT Vaccine Hesitancy Proposed topics: Messaging Matters Submitted use case for AWS, just got approval for large language models AIM LC - hesitancy for staff Current work - declinations Focus not just on gaps, but what NACHC has done to get vaccines Foundational Paper Final Meeting scheduling: Can check in 6/27 call for July calls (currently scheduled for 7/4 and 7/25)	@Raymonde Uy will be working with Noah next week for qualitative analysis Will work on methodology of analysisFocus on what
5/13/24 Internal Meeting		How do we want to present the activities? Step up view - less specific, maybe want to look at SOW deliverables, themes of project year @Pedro Carneiro (Deactivated) aiming to have draft of methods by 5/15 meeting PHIT working on partner mapping, resources on
4/26/24 Foundational Paper	Methodology & Results	Methodology Julia & Sarah can fill in the rest of the details More of a broad overview Table of interventions by year Results - what do we want to present? Talk about the update into activities by year - able to enroll x number of centers A little bit of Informatics - different sources with different quality, can it be used to create population level data Demographics table Interoperability, quality issues, data extraction issues → gaps in data Role of Clinovations - started from Y3 exit interviews into Y4 How people stayed engaged → need for Learning Community intervention Tables: High level interventions table separated by year Only have table for Y5, may be able to put together Y4 Lessons learned, quotes Enrollment table - retention and growth of participation over the years Informatics table - demographics	Clinovations team will put together short narrative on role Sarah & Pedro to meet in the next few weeks to put together bullet points on the rest of the paper
3/28/24 Foundational Paper	Review Intro & Methodology	Sarah sent CDC Project Determination to Joy We can receive no-cost extension through December, still would like to have done by July Asked to share work at NIC in August, ATSU potentially to co-present Methods May want to limit scope, so that we can limit results How was the learning community developed? Reached out to HCCNs, PCAs, how many people wanted to be part of it, how were they chosen, what did it mean to be part of the learning community, etc. Appendix with each year’s interventions Addition of Informatics team after first two years of project How a national organization can partner with the CDC and health centers to help support the safety net Results section plan: How can we bring people in to support that effort? Results from the learning community Quantitative data gathered Capacity building (documentation) Informatics data quality lens? Qualitative Surveys, feedback from participants, even attendance - what is the value added, why did they stick around, etc. Joy out the week of the 8th and 15th	Project Determination sent says pending, Sarah will send official letter Pedro & PHIT to discuss methods, will help inform results Week of April 22nd, NACHC to send over progress for ATSU to react to
3/7/24 Vaccine Hesitancy		@Raymonde Uy has been working on analysis of transcripts - proof of concept stage 23 or 24 text transcripts across multiple sessions/organizations Selected a few, de-identified, and ran through language model to extract themes and sub-themes, output also included quotes ATSU experience with machine learning? Atlas has AI now - https://atlasti.com/ Joy will reach out to other colleagues at other schools, head of IT Validation? Do we run it through two AI engines, have someone also analyze/verify? Grounded theory approach seems to fit in with AI language training Run through another model and compare the two? Emory has summary document of themes Bidirectional Encoder Representations from Transformers (BERT) https://stanfordnlp.github.io/CoreNLP/	Spreadsheet of data on vaccine hesitancy survey from Clinovations @Sarah Price will write summary of Messaging Matters methodology IRB - Sarah will send CDC project determination
2/22/24 Foundational Paper	Review updated outline Discuss data (Pedro) Potential journals Next steps, roles & responsibilities	Potential additional papers that ATSU could take lead on? Erin (background in microbiology) interested in increasing vaccination rates on particular microbe (pneumo, hep) Outline review Discussing more findings than impact (no baseline, not experimental model) - here are the observations we had from the project Discussion how are we going to define “effective”? More shots in arms, improved infrastructure Talk about and define impact here in Discussion section Conclusion Don’t want to overload, but can mention implications without going into great detail Journals Preventative Medicine Reports - published value transformation paper previously with Cheryl (https://www.sciencedirect.com/science/article/pii/S2211335519300737) Fits into aims & scope of journal: https://www.sciencedirect.com/journal/preventive-medicine-reports Submitted Nov, published in May (6 month process) Journal of Multidisciplinary Healthcare Journal of Public Health Management & Practice - previous immunization paper Advances in Health Sciences Education - Dovepress, if you submit and they turn it down, can suggest others Can compare impact factor Priority for open access journals ATSU librarian can investigate and check for predatory journals Can look for higher impact, higher level journal afterwards CDC involvement Amy has expressed interest in co-authoring Usually adds time for review, needs to go through CDC clearance Once it’s made it through, then good chance of publishing (like an additional layer of peer review); adds value	NACHC will internally discuss and propose additional opportunities for papers PHIT will send follow up email with manuscript list, other resources
2/1/24 - PHIT, Clinovations, Informatics, ATSU Vaccine Hesitancy	Brief Intros Review and discussion of content shared last month (in this folder: Vaccine Hesitancy Paper) Discussion around current work on Declinations and how that might fit in Next steps: Outline Additional data/content sources Roles/Responsibilities	Vaccine hesitancy interviews - 19 HCs, 90 minute interviews, 4 sessions of 60-90 minutes (1 facilitated discussion with staff, 1 with patients; reconvened for feedback) Where do these fit within pandemic context, changing messaging Healthcare worker mandate Survey that gets into what staff felt were the greatest challenges and strategies when it comes to addressing patient hesitancy Curt has done work on ranked order responses, will take a look 2 papers - patient & providers?	Sarah to start write up of Messaging Matters work PHIT & Clinovations to discuss general outline, sending responses to ATSU Pedro can take first pass and share a table send data files to Curt & Dev Sarah can send project determination materials to Joy to submit to IRB
1/25/24 - PHIT, Clinovations, Informatics, ATSU Foundational Paper	Updates? Feedback on materials Roles & responsibilities Next steps, future meeting schedule	Goal today: Check in on documents, create plan of attack Outline of Foundations Paper Adult Immunizations today Standards of immunization Here’s are the interventions offered to health centers, what they chose to focus on Here’s some data (not strong data to show improvement of rates, talk about the quality of the data?) What we learned from project, future policy implications What do we want from this paper? Telling the story of the project, vaccinations and quality improvement Look at intersectional groups' vaccination rates Limited SDOH data - age, sex, gender identity, race, region, federal poverty level, insurance status Do we have data from CDC for non-HC vaccination? Informatics was not able to find good comparison data https://www.cdc.gov/flu/fluvaxview/dashboard/vaccination-adult-coverage.html Bigger story that HCs take care of the most vulnerable, we gave them options of what to do, how they implemented Include data anecdotally, trends Pneumo interventions more effective in Y5, flu & COVID hit by vaccine hesitancy Health centers got to pick interventions, show which ones picked what and what we know what they did over the project years and can talk about individual interventions in later papers? Winding Waters & PneumoRecs app intervention Holyoke outreach to Spanish speaking patients and making phone calls Shingrix could be its own paper later, what happens when policy changes Overarching theme - foundational principles of standards of adult immunization Where can we submit this? Previous vaccination paper submitted to Journal of Public Health Management & Practice Good to work with Just published with MH in Journal of Healthcare Quality, solid journal and straightforward process BMC Public Health Springer, International Health? May be easier to decide once we have written in	NACHC team to: narrow down interventions work on outline Feedback on what we can present data-wise
12/15/23 – PHIT, Clinovations, Informatics, ATSU	Agenda Introductions Continuation of Foundational Paper conversation Overview of Vaccine Hesitancy Paper Roles & Responsibilities Next steps Meeting schedule & cadence	Joy has two other team members that would like to be involved, will be on future calls Review of COVID data Want to focus on flu & COVID - affect general population as opposed to Pneumo, more difficult to set up denominators Intervention table review https://link.springer.com/article/10.1007/s10900-020-00858-0 Table that shows support tasks with health centers ATSU still going through resources, will have more feedback later on Future meeting schedule: Once a month Decide how many papers to work on at once: 2? Hesitancy & foundational; qualitative could be another separate paper?	Work on grouping/categorizing intervention table, add reflections/quotes Will send another doodle poll for availability
11/30/23 - PHIT, Clinovations, Informatics, ATSU	Introductions Data and Intervention Review Roles/Responsibilities Next Steps	Gaps in the data: Denominator: number of health centers involved (Y5 has larger denominator but lower rate. Look at flu data at a jurisdiction level? Need to look at 63% jurisdiction (NW/NE, etc..) missing. Success - not looking at overall populations but target population rates and the interventions Next meeting - will split time to discuss foundational & vaccine hesitancy papers	Clinovations to add to interventions table TBD Sarah to send upload NACHC to share documents, SharePoint folder (update with data, interventions, final reports)
11/13/23 - PHIT, Informatics, ATSU	Introductions –2 min Review of Manuscript List: Publications: 5 min Discussion –led by Joy Lewis: 20 min Establish next steps: 18 min File sharing—new Sharepoint link/folders: Publications Roles/Responsibilities Timeline (via Asana)	Foundational paper and Vaccine Hesitancy can be written concurrently and highest interest 2 & 5 as next Form workgroups Usual cadence is once a month with additional as needed Joy more big picture, editing Can bring in others for background, other work We have IRB, would just need to submit to ATSU to get it signed off	Next meeting right after Thanksgiving, want to get started before winter break - set up Doodle poll
11/2/23 - Internal + Clinovations		Joy “We are in, we are pumped, looking forward to the meeting!” Clinovations proposal for work for this year: Gaps found by Clinovations? See section 6 IIS - broader policy landscape at play regardless of responses from partners Group - Data quality, validation and population health Policy lens - Lack of clarity around hard recommendations and soft recommendations - evidence and guidance layer Policy - ability to ensure all the infrastructure gets built - state infrastructures of various quality and nothing that connects them and compels public health to be consistent Access & supply chain Connectivity and services between public health and HIE’s, tracking vaccines, who has what, doing the dashboard, getting patients in, vaccine hesitancy Vaccine hesitancy Was there an intervention done? AC/HCN did motivational interviewing intervention Matches up with Messaging Matters work done before Haven’t explored much in the data, but we may have some on patient reason for decline and could match with provider (Jen’s idea) Vaccine hesitancy billing codes - could include in one of the informational sessions this year Dashboard & population health management What is real data vs. UDS data?	Clinovations will send updated workplan proposal, NACHC team will react to
10/19/23 - Internal	Discussion Points: Reflections Slide deck data pitch Next Steps Data ATSU AAPCHO CDC	1 pager pitch to Kyu The “why” - we want these to represent what our findings will be for other reports, publications What gaps in literature exist where our data can be useful? What do other partners see? Conversation with AAPCHO Most concrete conversation we’ve had so far - idea of tracking transportation and diabetes patients Already has data infrastructure in place Future project Rosy has written articles on this topic Publications Idea - Put together quant & qual tables for frequencies, themes Send to partners to get their feedback and see what they want us to publish We need partners to be working on it as well Kyu interested in volume Numbers and reach also have an impact, important to showcase Tell the story at the high level may be less impactful, should do both at the same time	Need to answer the question - what data is working? Slide deck data pitch Next meeting with Jim McRay at team Meeting with Joy soon PHIT will set up work session Pedro will do some background research
10/12/23 - Internal	Debrief Meeting with Joy and ASTU Minutes from Meeting Roles and Responsibilities moving forward: Current data and year 6 Prepare for Amy meeting on Friday Year 6 Data Plan Sarah and Jen Publishing strategy Next steps Agenda next meeting	Last meeting was positive but overwhelming Suggestion: come up with one project, writing first paper? Rely on Joy’s expertise Y6 - Micro project to clean the data? Two data angles: Write about Informatics - quality of EMR data? Two stories: quality of data received, structure can be very different Study - get sample of data Possible topic: uptake of flu vaccines before, during, and after pandemic Mixed-methods and include vaccine hesitancy? Workplan has 3 publications Papers should complement each other One page on findings and how they relate to each other New models of care - pharmacist What do we have? Pharmacy (HC) data, intervention, building the business case project Policy - what do the policy people want to know? CDC priority - national adult vaccine program Depending on the journal, can divvy up authorship, writing Literature review Can we use ATSU as resource to access?	Sarah will send updates from policy team Next meeting with Joy, will send her our list
10/10/23 - ATSU	Purpose: To develop a strategy to publish the Immunization Data from year 5 and 6 of the Adult Immunization work Welcome and Introductions (3 minutes) Adult Immunization Overview (5 minutes) 5 years in the making… Data Overview Jen (5 minutes) the what the where the mess Clinovations Findings (3 m) Publication Strategy 30 minutes QI Data Qualitative feedback COVID? SDOH? Next Steps	Able to collect information on health center patients who got vaccines elsewhere? Historical vaccine information is recorded, likely incomplete 3 buckets of data - infrastructure (Informatics, what’s in the EMR), sample selection, implementation 60+ hours of qualitative data on distrust of healthcare system, interventions to minimize (Messaging Matters, Emory has transcripts? Recordings on Zoom/Confluence) Lack of ability to communicate scientific information to the public - what we knew and what level we knew it Data can illustrate what we can do better “If I understood what you meant, would my face look like this” book SDOH - data is limited, haven’t done much analysis - best is housing and still <50% Primary language has some data Initial ideas Positive outliers - short papers to give examples of what health centers are doing as successful interventions Bigger picture - Vaccination capabilities (as opposed to capacity) Advocate for national system of adult immunization (similar to VFC) Compare by state and presence/absence of adult immunization system and % of patients that got vaccinated Ability of states to who have data or not Qualitative - asking facilitators HC data where they are the only provider in their county? Data is in the hands of patients If there were to be a national vaccine program instituted next year, would health centers be ready? No, many operational factors, policy factors We have CDC IRB/Project Determination	Joy will think about who else may be interested Shared document with spreadsheets with topic, roles
Clinovations - Potential Publication Topics Utilizing the Standards to Reshape Vaccine Programs Audience: Clinical leadership, health center administrators Focus: Experience and benefits of utilizing pharmacists to expand access to adult immunizations Leveraging Health Center Staff to Expand Access Audience: Pharmacy, Primary Care Focus: Experience and benefits of utilizing pharmacists to expand access to adult immunizations Assessment of the Standards for Adult Immunization Practice Audience: Clinical leadership, health center administrators, national policy Focus: Reflection that the Standards address “internal” factors that health centers can control, however, there are “external” factors (e.g., workforce and scope of work policy; reimbursement levels; availability and quality of external data) that impact the success of adult immunization programs Future topics to consider, based on how Year 6 work unfolds with opportunities to incorporate publication plans into the partnerships and assessment tools and/or surveys Addressing Vaccine Hesitancy Audience: Clinical administrators Focus: Determine an impact on immunization rates by supporting health center staff address vaccine hesitancy (Set up pre- and post- intervention surveys and data reports) Automating Compliance with Clinical Guidelines Audience: Informatics Focus: Determine an impact on immunization rates by configuring health IT functionality (e.g., CDS, standing orders) to improve use
10/5/23 - Internal	Goal: To design the data strategy for Adult Immunizations Current state: Jen Data Elements needed: Sarah Data Analyses and Visualization: Jen and Pedro Manuscripts	Focusing this year on publishing We have final report from Clinovations as springboard Working with Joy who will provide support Kathy received gaps in literature from CDC Policy lens? Position policy paper? Should aim to have a quantitative and qualitative; mixed-methods paper later on What data elements are robust enough? What variables do we have and what variables can be created? Subpopulations? ATSU can do some of the analysis and validation of quantitative data Connect with COVID data? Characteristics of patients and lack of quantified SDOH? Challenges with implementing policy without that data Also looking at revisiting white paper on adult immunization policy Could reference quality measures - collaboration between vaccines and pregnancy? Infectious disease - flu, COVID, & RSV all forecasted to be moderate, worry of zip codes where these are converging Needed: data strategy for Y6 AMY (CDC) Top Requests: Manuscripts Data, Data, Data Microlearnings for Adult Immunization Implementation (Package for CHCS) Bundle services (Hypertension, Pregnant women, OUD, SUD etc) Policy Paper to share lessons learned (ONC, Hill) Toolkits for scane and spread ( CEOS, PCAs, HCCNs, teams, Board) Follow-up with NACCHO to align the work	NACHC team: Define what is the story Areas of focus: Sarah and PHIT Team (plus Clinovations): Qualitative Jen/Pedro: Quantitative (immunization data) + some qualitative Primary authors: Sarah and Jen People/processes/technology/policy lens Data strategy for Y6: Jen to design what data we should focus on for subpopulations For Joy meeting on Tuesday, 10/10: Jen to run through the data context, Sarah to run through qualitative