Real-time brand sentiment analysis with ClickHouse and Redpanda

link管理

链接快照平台

输入网页链接，自动生成快照
标签化管理网页链接

相关文章推荐

慷慨的熊猫 · Hibernate Community • ...· 昨天 ·

英姿勃勃的麦片 · 回顾：“新雅·古典学工作坊”第二期活动纪要- ...· 2 月前 ·

玩手机的饭卡 · 轨道上的大湾区建设驶入快车道-南方日报· 2 月前 ·

虚心的炒面 · 如何从sqlalchemy中另一个目录的模型 ...· 3 月前 ·

有情有义的汉堡包 · 中国石油大学(北京)克拉玛依校区石油学院· 4 月前 ·

刀枪不入的马铃薯 · 解决 springboot + JPA + ...· 4 月前 ·

</noscript><div id="__next"><div class="styles_Layout__QUZWZ" style="padding-top:0"><div class="styles_Header__nP6fi"><div class="styles_Header__banner__6l2bu"><p>BREAKING NEWS: Redpanda acquires Benthos &amp; introduces Redpanda Connect with 220+ connectors. <a href="/blog/redpanda-connect">Learn more</a>.</p></div><header role="contentinfo" class="styles_DesktopHeader__ITMeV"><nav class="styles_DesktopHeader__desktopNavTop__GbKau"><ul class="styles_DesktopHeader__navListTop__sGeTQ"><li class="styles_DesktopHeader__navItem___cn2G"><a class="styles_DesktopHeader__navButton__svyeJ" target="_blank" href="http://cloud.redpanda.com/">CLOUD LOGIN</a></li><li class="styles_DesktopHeader__navItem___cn2G"><a class="styles_DesktopHeader__navButton__svyeJ" target="_blank" href="https://support.redpanda.com/hc/en-us">SUPPORT</a></li><li class="styles_DesktopHeader__navItem___cn2G"><a class="styles_DesktopHeader__socialLink__2Zi5d styles_DesktopHeader__navButton__svyeJ" target="_blank" rel="noopener noreferrer" href="https://redpanda.com/slack"><img class="styles_DesktopHeader__socialIcon__nwRg9" src="https://images.ctfassets.net/paqvtpyf8rwu/69HyyrugepthrFF25coB86/bbc52835d1aa2c5714b30f62a957133a/slack__1_.svg" alt="Slack (homepage header)" width="30" height="30"/> </a></li><li class="styles_DesktopHeader__navItem___cn2G"><a class="styles_DesktopHeader__socialLink__2Zi5d styles_DesktopHeader__navButton__svyeJ" target="_blank" rel="noopener noreferrer" href="https://github.com/redpanda-data/redpanda/"><img class="styles_DesktopHeader__socialIcon__nwRg9" src="https://images.ctfassets.net/paqvtpyf8rwu/6ZoZUb18LuxNlX5qyhD1vI/08a798d68104fe6a7f9b14d7251a45a3/github__1_.svg" alt="Github" width="30" height="30"/>8.1K</a></li></ul></nav><nav class="styles_DesktopHeader__desktopNavBottom__T_AZD"><ul class="styles_DesktopHeader__navListBottom__Rgvzi"><a class="styles_DesktopHeader__logo__gpnyQ" aria-label="Red Panda Logo" href="/"><img src="https://images.ctfassets.net/paqvtpyf8rwu/5wODDUzLP7teOGDS7M5m39/466f646e44c2ccc4e78236b17d7d68ac/Logo.svg" alt="" title="Redpanda Logo" width="155" height="36"/></a><li class="styles_DesktopHeader__navItem___cn2G" aria-label="Toggle PRODUCTS subnav dropdown"><div><button class="styles_DesktopHeader__navButton__svyeJ">PRODUCTS<span><svg width="12" height="7" viewbox="0 0 12 7" fill="none" xmlns="http://www.w3.org/2000/svg" class=""><path d="M1 1L6 6L11 1" stroke="#667085" stroke-width="1.67" stroke-linecap="round" stroke-linejoin="round"/></svg></span></button><div class="styles_DesktopHeader__subNav__LQcM5"><ul class="styles_DesktopHeader__subNavList___F8Zs"><li class="styles_DesktopHeader__subNavItem__ZFWTy"><a class="styles_Header__subNavLink__qf_Y6" target="_self" rel="" href="/what-is-redpanda"><div class="styles_Header__subNavLinkTextAndIcon___JAkR"><img class="styles_Header__subNavLinkIcon__F4K99" src="https://images.ctfassets.net/paqvtpyf8rwu/5VKIqaX8JhLWcL9mPUnpXg/019d6f9d8594c781ab0cb4401b7c45a7/r-icon.svg" alt="Overview"/><div><div class="styles_Header__subNavLinkGroup___MYi3"><p class="styles_Header__subNavLinkText__dWweB">Overview</p><div class="styles_Header__subNavLinkArrow__3zt7q"><svg width="17" height="13" viewbox="0 0 17 16" fill="none" xmlns="http://www.w3.org/2000/svg"><path d="M10.7077 10.313L9.42138 9.02034L12.4023 6.05306L-6.79749e-05 6.04121L0.00175553 4.218L12.3704 4.22985L9.43961 1.28536L10.7314 -2.249e-07L15.8765 5.16881L10.7077 10.313Z" fill="#E2401B"/></svg></div></div><span class="styles_Header__subNavLinkSubText___ytDQ">Technical tour of the Redpanda platform</span></div></div></a></li><li class="styles_DesktopHeader__subNavItem__ZFWTy"><a class="styles_Header__subNavLink__qf_Y6" target="_self" rel="" href="/platform-capabilities"><div class="styles_Header__subNavLinkTextAndIcon___JAkR"><img class="styles_Header__subNavLinkIcon__F4K99" src="https://images.ctfassets.net/paqvtpyf8rwu/3QzpsfG9Ba6u6Zu14Zmb0X/06873b28b5459a166879833affc7923d/capabilities.svg" alt="Core capabilities"/><div><div class="styles_Header__subNavLinkGroup___MYi3"><p class="styles_Header__subNavLinkText__dWweB">Core capabilities</p><div class="styles_Header__subNavLinkArrow__3zt7q"><svg width="17" height="13" viewbox="0 0 17 16" fill="none" xmlns="http://www.w3.org/2000/svg"><path d="M10.7077 10.313L9.42138 9.02034L12.4023 6.05306L-6.79749e-05 6.04121L0.00175553 4.218L12.3704 4.22985L9.43961 1.28536L10.7314 -2.249e-07L15.8765 5.16881L10.7077 10.313Z" fill="#E2401B"/></svg></div></div><span class="styles_Header__subNavLinkSubText___ytDQ">Explore all features and functions</span></div></div></a></li><li class="styles_DesktopHeader__subNavItem__ZFWTy"><a class="styles_Header__subNavLink__qf_Y6" target="_self" rel="" href="/connect"><div class="styles_Header__subNavLinkTextAndIcon___JAkR"><img class="styles_Header__subNavLinkIcon__F4K99" src="https://images.ctfassets.net/paqvtpyf8rwu/5lvkF4LBIDOTT5fXPzb2M3/98ec3cc6836193f2009004d2b7b0930c/rc-icon-menu.svg" alt="Redpanda Connect"/><div><div class="styles_Header__subNavLinkGroup___MYi3"><p class="styles_Header__subNavLinkText__dWweB">Redpanda Connect</p><div class="styles_Header__subNavLinkArrow__3zt7q"><svg width="17" height="13" viewbox="0 0 17 16" fill="none" xmlns="http://www.w3.org/2000/svg"><path d="M10.7077 10.313L9.42138 9.02034L12.4023 6.05306L-6.79749e-05 6.04121L0.00175553 4.218L12.3704 4.22985L9.43961 1.28536L10.7314 -2.249e-07L15.8765 5.16881L10.7077 10.313Z" fill="#E2401B"/></svg></div></div><span class="styles_Header__subNavLinkSubText___ytDQ">Stream processing and connection framework</span></div></div></a></li><li class="styles_DesktopHeader__subNavItem__ZFWTy"><a class="styles_Header__subNavLink__qf_Y6" target="_self" rel="" href="/redpanda-console-kafka-ui"><div class="styles_Header__subNavLinkTextAndIcon___JAkR"><img class="styles_Header__subNavLinkIcon__F4K99" src="https://images.ctfassets.net/paqvtpyf8rwu/2JdeLBOezvJgJ1SNId6FFZ/ba1a95da69557a00cb7c792e862f97ba/console.svg" alt="Redpanda Console"/><div><div class="styles_Header__subNavLinkGroup___MYi3"><p class="styles_Header__subNavLinkText__dWweB">Redpanda Console</p><div class="styles_Header__subNavLinkArrow__3zt7q"><svg width="17" height="13" viewbox="0 0 17 16" fill="none" xmlns="http://www.w3.org/2000/svg"><path d="M10.7077 10.313L9.42138 9.02034L12.4023 6.05306L-6.79749e-05 6.04121L0.00175553 4.218L12.3704 4.22985L9.43961 1.28536L10.7314 -2.249e-07L15.8765 5.16881L10.7077 10.313Z" fill="#E2401B"/></svg></div></div><span class="styles_Header__subNavLinkSubText___ytDQ">An interactive web UI for Kafka</span></div></div></a></li><li class="styles_DesktopHeader__subNavItem__ZFWTy"><a class="styles_Header__subNavLink__qf_Y6" target="_self" rel="" href="/compare-platform-editions"><div class="styles_Header__subNavLinkTextAndIcon___JAkR"><img class="styles_Header__subNavLinkIcon__F4K99" src="https://images.ctfassets.net/paqvtpyf8rwu/5xWZuYA6eAr8uVmfC8GpnX/26a09a1b162c0878cddd4852b3ba4023/comparison.svg" alt="Compare editions"/><div><div class="styles_Header__subNavLinkGroup___MYi3"><p class="styles_Header__subNavLinkText__dWweB">Compare editions</p><div class="styles_Header__subNavLinkArrow__3zt7q"><svg width="17" height="13" viewbox="0 0 17 16" fill="none" xmlns="http://www.w3.org/2000/svg"><path d="M10.7077 10.313L9.42138 9.02034L12.4023 6.05306L-6.79749e-05 6.04121L0.00175553 4.218L12.3704 4.22985L9.43961 1.28536L10.7314 -2.249e-07L15.8765 5.16881L10.7077 10.313Z" fill="#E2401B"/></svg></div></div><span class="styles_Header__subNavLinkSubText___ytDQ">Understand Redpanda Enterprise &amp; Community Editions</span></div></div></a></li></ul></div></div></li><li class="styles_DesktopHeader__navItem___cn2G" aria-label="Toggle CLOUD subnav dropdown"><div><button class="styles_DesktopHeader__navButton__svyeJ">CLOUD<span><svg width="12" height="7" viewbox="0 0 12 7" fill="none" xmlns="http://www.w3.org/2000/svg" class=""><path d="M1 1L6 6L11 1" stroke="#667085" stroke-width="1.67" stroke-linecap="round" stroke-linejoin="round"/></svg></span></button><div class="styles_DesktopHeader__subNav__LQcM5"><ul class="styles_DesktopHeader__subNavList___F8Zs"><li class="styles_DesktopHeader__subNavItem__ZFWTy"><a class="styles_Header__subNavLink__qf_Y6" target="_self" rel="" href="/redpanda-cloud/"><div class="styles_Header__subNavLinkTextAndIcon___JAkR"><img class="styles_Header__subNavLinkIcon__F4K99" src="https://images.ctfassets.net/paqvtpyf8rwu/2SnuVKCsTZm7ka8dpwMaDt/3ea7a928ad562df2ebfec627418eab0f/cloud-overview.svg" alt="Cloud Overview"/><div><div class="styles_Header__subNavLinkGroup___MYi3"><p class="styles_Header__subNavLinkText__dWweB">Cloud Overview</p><div class="styles_Header__subNavLinkArrow__3zt7q"><svg width="17" height="13" viewbox="0 0 17 16" fill="none" xmlns="http://www.w3.org/2000/svg"><path d="M10.7077 10.313L9.42138 9.02034L12.4023 6.05306L-6.79749e-05 6.04121L0.00175553 4.218L12.3704 4.22985L9.43961 1.28536L10.7314 -2.249e-07L15.8765 5.16881L10.7077 10.313Z" fill="#E2401B"/></svg></div></div><span class="styles_Header__subNavLinkSubText___ytDQ">Learn about all the options for a fully managed services </span></div></div></a></li><li class="styles_DesktopHeader__subNavItem__ZFWTy"><a class="styles_Header__subNavLink__qf_Y6" target="_self" rel="" href="/redpanda-cloud/serverless"><div class="styles_Header__subNavLinkTextAndIcon___JAkR"><img class="styles_Header__subNavLinkIcon__F4K99" src="https://images.ctfassets.net/paqvtpyf8rwu/7mPS6G6ZFqP8fegOFUZ3Fh/c273f3d151a24bbb0f98ab0c96e51101/serverless.svg" alt="Redpanda Serverless"/><div><div class="styles_Header__subNavLinkGroup___MYi3"><p class="styles_Header__subNavLinkText__dWweB">Redpanda Serverless</p><div class="styles_Header__subNavLinkArrow__3zt7q"><svg width="17" height="13" viewbox="0 0 17 16" fill="none" xmlns="http://www.w3.org/2000/svg"><path d="M10.7077 10.313L9.42138 9.02034L12.4023 6.05306L-6.79749e-05 6.04121L0.00175553 4.218L12.3704 4.22985L9.43961 1.28536L10.7314 -2.249e-07L15.8765 5.16881L10.7077 10.313Z" fill="#E2401B"/></svg></div></div><span class="styles_Header__subNavLinkSubText___ytDQ">Explore Redpanda Serverless, the fastest way to get started</span></div></div></a></li></ul></div></div></li><li class="styles_DesktopHeader__navItem___cn2G" aria-label="Toggle CUSTOMERS subnav dropdown"><a class="styles_DesktopHeader__navButton__svyeJ" href="/customers">CUSTOMERS</a></li><li class="styles_DesktopHeader__navItem___cn2G" aria-label="Toggle LEARN subnav dropdown"><div><button class="styles_DesktopHeader__navButton__svyeJ">LEARN<span><svg width="12" height="7" viewbox="0 0 12 7" fill="none" xmlns="http://www.w3.org/2000/svg" class=""><path d="M1 1L6 6L11 1" stroke="#667085" stroke-width="1.67" stroke-linecap="round" stroke-linejoin="round"/></svg></span></button><div class="styles_DesktopHeader__subNav__LQcM5"><ul class="styles_DesktopHeader__subNavList___F8Zs"><li class="styles_DesktopHeader__subNavItem__ZFWTy"><a class="styles_Header__subNavLink__qf_Y6" target="_blank" rel="noopener noreferrer" href="https://docs.redpanda.com/docs/home"><div class="styles_Header__subNavLinkTextAndIcon___JAkR"><img class="styles_Header__subNavLinkIcon__F4K99" src="https://images.ctfassets.net/paqvtpyf8rwu/2pOpTO3vvTj6lfkIgOzfqU/9e34792be89009df7cb22f4f599c8f00/docs-icon.svg" alt="Documentation"/><div><div class="styles_Header__subNavLinkGroup___MYi3"><p class="styles_Header__subNavLinkText__dWweB">Documentation</p><div class="styles_Header__subNavLinkArrow__3zt7q"><svg width="17" height="13" viewbox="0 0 17 16" fill="none" xmlns="http://www.w3.org/2000/svg"><path d="M10.7077 10.313L9.42138 9.02034L12.4023 6.05306L-6.79749e-05 6.04121L0.00175553 4.218L12.3704 4.22985L9.43961 1.28536L10.7314 -2.249e-07L15.8765 5.16881L10.7077 10.313Z" fill="#E2401B"/></svg></div></div><span class="styles_Header__subNavLinkSubText___ytDQ">Dive deep into features and APIs.</span></div></div></a></li><li class="styles_DesktopHeader__subNavItem__ZFWTy"><a class="styles_Header__subNavLink__qf_Y6" target="_self" rel="" href="/blog"><div class="styles_Header__subNavLinkTextAndIcon___JAkR"><img class="styles_Header__subNavLinkIcon__F4K99" src="https://images.ctfassets.net/paqvtpyf8rwu/iSxaUjmTDtKvvvwLY7Hz8/f5583a607ffbde9009f9cc9cac825cad/blog-alt.svg" alt="Blog"/><div><div class="styles_Header__subNavLinkGroup___MYi3"><p class="styles_Header__subNavLinkText__dWweB">Blog</p><div class="styles_Header__subNavLinkArrow__3zt7q"><svg width="17" height="13" viewbox="0 0 17 16" fill="none" xmlns="http://www.w3.org/2000/svg"><path d="M10.7077 10.313L9.42138 9.02034L12.4023 6.05306L-6.79749e-05 6.04121L0.00175553 4.218L12.3704 4.22985L9.43961 1.28536L10.7314 -2.249e-07L15.8765 5.16881L10.7077 10.313Z" fill="#E2401B"/></svg></div></div><span class="styles_Header__subNavLinkSubText___ytDQ">In-depth posts on streaming data topics.</span></div></div></a></li><li class="styles_DesktopHeader__subNavItem__ZFWTy"><a class="styles_Header__subNavLink__qf_Y6" target="_blank" rel="noopener noreferrer" href="https://university.redpanda.com"><div class="styles_Header__subNavLinkTextAndIcon___JAkR"><img class="styles_Header__subNavLinkIcon__F4K99" src="https://images.ctfassets.net/paqvtpyf8rwu/78Ji8ECFrOzxOqvMWwYdy/c5a3551114aff387f238a2720326f40d/redpanda-u.svg" alt="Redpanda University"/><div><div class="styles_Header__subNavLinkGroup___MYi3"><p class="styles_Header__subNavLinkText__dWweB">Redpanda University</p><div class="styles_Header__subNavLinkArrow__3zt7q"><svg width="17" height="13" viewbox="0 0 17 16" fill="none" xmlns="http://www.w3.org/2000/svg"><path d="M10.7077 10.313L9.42138 9.02034L12.4023 6.05306L-6.79749e-05 6.04121L0.00175553 4.218L12.3704 4.22985L9.43961 1.28536L10.7314 -2.249e-07L15.8765 5.16881L10.7077 10.313Z" fill="#E2401B"/></svg></div></div><span class="styles_Header__subNavLinkSubText___ytDQ">Guided courses on building real-time apps.</span></div></div></a></li><li class="styles_DesktopHeader__subNavItem__ZFWTy"><a class="styles_Header__subNavLink__qf_Y6" target="_self" rel="" href="/events"><div class="styles_Header__subNavLinkTextAndIcon___JAkR"><img class="styles_Header__subNavLinkIcon__F4K99" src="https://images.ctfassets.net/paqvtpyf8rwu/3Tq6HBZ1W3blN9ht3otKir/fdd4b4a56218173c4199424d1f4eb776/events.svg" alt="Upcoming events"/><div><div class="styles_Header__subNavLinkGroup___MYi3"><p class="styles_Header__subNavLinkText__dWweB">Upcoming events</p><div class="styles_Header__subNavLinkArrow__3zt7q"><svg width="17" height="13" viewbox="0 0 17 16" fill="none" xmlns="http://www.w3.org/2000/svg"><path d="M10.7077 10.313L9.42138 9.02034L12.4023 6.05306L-6.79749e-05 6.04121L0.00175553 4.218L12.3704 4.22985L9.43961 1.28536L10.7314 -2.249e-07L15.8765 5.16881L10.7077 10.313Z" fill="#E2401B"/></svg></div></div><span class="styles_Header__subNavLinkSubText___ytDQ">Join our live virtual events to learn from experts.</span></div></div></a></li><li class="styles_DesktopHeader__subNavItem__ZFWTy"><a class="styles_Header__subNavLink__qf_Y6" target="_self" rel="" href="/resources"><div class="styles_Header__subNavLinkTextAndIcon___JAkR"><img class="styles_Header__subNavLinkIcon__F4K99" src="https://images.ctfassets.net/paqvtpyf8rwu/5tRzRjoeG3Ny4TwWaF1WAv/41485d30c174cd9753e00d0cb34d3354/resources.svg" alt="On-demand resources"/><div><div class="styles_Header__subNavLinkGroup___MYi3"><p class="styles_Header__subNavLinkText__dWweB">On-demand resources</p><div class="styles_Header__subNavLinkArrow__3zt7q"><svg width="17" height="13" viewbox="0 0 17 16" fill="none" xmlns="http://www.w3.org/2000/svg"><path d="M10.7077 10.313L9.42138 9.02034L12.4023 6.05306L-6.79749e-05 6.04121L0.00175553 4.218L12.3704 4.22985L9.43961 1.28536L10.7314 -2.249e-07L15.8765 5.16881L10.7077 10.313Z" fill="#E2401B"/></svg></div></div><span class="styles_Header__subNavLinkSubText___ytDQ">Tech talks, streamcasts, reports, and more.</span></div></div></a></li><li class="styles_DesktopHeader__subNavItem__ZFWTy"><a class="styles_Header__subNavLink__qf_Y6" target="_self" rel="" href="https://redpanda.com/slack"><div class="styles_Header__subNavLinkTextAndIcon___JAkR"><img class="styles_Header__subNavLinkIcon__F4K99" src="https://images.ctfassets.net/paqvtpyf8rwu/PMcVPLOUI853NUus1Ofqy/41306403501d2129761387a0291cc60d/slack.svg" alt="Community Slack"/><div><div class="styles_Header__subNavLinkGroup___MYi3"><p class="styles_Header__subNavLinkText__dWweB">Community Slack</p><div class="styles_Header__subNavLinkArrow__3zt7q"><svg width="17" height="13" viewbox="0 0 17 16" fill="none" xmlns="http://www.w3.org/2000/svg"><path d="M10.7077 10.313L9.42138 9.02034L12.4023 6.05306L-6.79749e-05 6.04121L0.00175553 4.218L12.3704 4.22985L9.43961 1.28536L10.7314 -2.249e-07L15.8765 5.16881L10.7077 10.313Z" fill="#E2401B"/></svg></div></div><span class="styles_Header__subNavLinkSubText___ytDQ">Ask our dev community your burning questions.</span></div></div></a></li><li class="styles_DesktopHeader__subNavItem__ZFWTy"><a class="styles_Header__subNavLink__qf_Y6" target="_blank" rel="noopener noreferrer" href="https://docs.redpanda.com/redpanda-labs/"><div class="styles_Header__subNavLinkTextAndIcon___JAkR"><img class="styles_Header__subNavLinkIcon__F4K99" src="https://images.ctfassets.net/paqvtpyf8rwu/2sKZ7Ww3SjgT8WvbCjwqmJ/3596a4779f72899ed09c4fbb31ed30c3/labs.svg" alt="Redpanda Labs"/><div><div class="styles_Header__subNavLinkGroup___MYi3"><p class="styles_Header__subNavLinkText__dWweB">Redpanda Labs</p><div class="styles_Header__subNavLinkArrow__3zt7q"><svg width="17" height="13" viewbox="0 0 17 16" fill="none" xmlns="http://www.w3.org/2000/svg"><path d="M10.7077 10.313L9.42138 9.02034L12.4023 6.05306L-6.79749e-05 6.04121L0.00175553 4.218L12.3704 4.22985L9.43961 1.28536L10.7314 -2.249e-07L15.8765 5.16881L10.7077 10.313Z" fill="#E2401B"/></svg></div></div><span class="styles_Header__subNavLinkSubText___ytDQ">Examples, experiments, and inspiration.</span></div></div></a></li></ul></div></div></li><li class="styles_DesktopHeader__navItem___cn2G" aria-label="Toggle COMPANY subnav dropdown"><div><button class="styles_DesktopHeader__navButton__svyeJ">COMPANY<span><svg width="12" height="7" viewbox="0 0 12 7" fill="none" xmlns="http://www.w3.org/2000/svg" class=""><path d="M1 1L6 6L11 1" stroke="#667085" stroke-width="1.67" stroke-linecap="round" stroke-linejoin="round"/></svg></span></button><div class="styles_DesktopHeader__subNav__LQcM5"><ul class="styles_DesktopHeader__subNavList___F8Zs"><li class="styles_DesktopHeader__subNavItem__ZFWTy"><a class="styles_Header__subNavLink__qf_Y6" target="_self" rel="" href="/about-us/"><div class="styles_Header__subNavLinkTextAndIcon___JAkR"><img class="styles_Header__subNavLinkIcon__F4K99" src="https://images.ctfassets.net/paqvtpyf8rwu/5y3RsyLnrJb66KaVzyZ91d/d095abcc315fc62cd052936996a2eae3/about-icon.svg" alt="About us"/><div><div class="styles_Header__subNavLinkGroup___MYi3"><p class="styles_Header__subNavLinkText__dWweB">About us</p><div class="styles_Header__subNavLinkArrow__3zt7q"><svg width="17" height="13" viewbox="0 0 17 16" fill="none" xmlns="http://www.w3.org/2000/svg"><path d="M10.7077 10.313L9.42138 9.02034L12.4023 6.05306L-6.79749e-05 6.04121L0.00175553 4.218L12.3704 4.22985L9.43961 1.28536L10.7314 -2.249e-07L15.8765 5.16881L10.7077 10.313Z" fill="#E2401B"/></svg></div></div><span class="styles_Header__subNavLinkSubText___ytDQ">Meet our team and investors.</span></div></div></a></li><li class="styles_DesktopHeader__subNavItem__ZFWTy"><a class="styles_Header__subNavLink__qf_Y6" target="_self" rel="" href="/press"><div class="styles_Header__subNavLinkTextAndIcon___JAkR"><img class="styles_Header__subNavLinkIcon__F4K99" src="https://images.ctfassets.net/paqvtpyf8rwu/6JJ0u06zAS2yPM56Waf7hp/a5e226ddac7245f6e43a8d8ce446f85e/press.svg" alt="Press"/><div><div class="styles_Header__subNavLinkGroup___MYi3"><p class="styles_Header__subNavLinkText__dWweB">Press</p><div class="styles_Header__subNavLinkArrow__3zt7q"><svg width="17" height="13" viewbox="0 0 17 16" fill="none" xmlns="http://www.w3.org/2000/svg"><path d="M10.7077 10.313L9.42138 9.02034L12.4023 6.05306L-6.79749e-05 6.04121L0.00175553 4.218L12.3704 4.22985L9.43961 1.28536L10.7314 -2.249e-07L15.8765 5.16881L10.7077 10.313Z" fill="#E2401B"/></svg></div></div><span class="styles_Header__subNavLinkSubText___ytDQ">Check out the media buzz.</span></div></div></a></li><li class="styles_DesktopHeader__subNavItem__ZFWTy"><a class="styles_Header__subNavLink__qf_Y6" target="_self" rel="" href="/careers"><div class="styles_Header__subNavLinkTextAndIcon___JAkR"><img class="styles_Header__subNavLinkIcon__F4K99" src="https://images.ctfassets.net/paqvtpyf8rwu/7wxiMVSCkCzeIlskXUh02h/5c66960f5ae6a43634e68872d64ca243/careers.svg" alt="Careers"/><div><div class="styles_Header__subNavLinkGroup___MYi3"><p class="styles_Header__subNavLinkText__dWweB">Careers</p><div class="styles_Header__subNavLinkArrow__3zt7q"><svg width="17" height="13" viewbox="0 0 17 16" fill="none" xmlns="http://www.w3.org/2000/svg"><path d="M10.7077 10.313L9.42138 9.02034L12.4023 6.05306L-6.79749e-05 6.04121L0.00175553 4.218L12.3704 4.22985L9.43961 1.28536L10.7314 -2.249e-07L15.8765 5.16881L10.7077 10.313Z" fill="#E2401B"/></svg></div></div><span class="styles_Header__subNavLinkSubText___ytDQ">Explore Redpanda opportunities and culture.</span></div></div></a></li></ul></div></div></li><li class="styles_DesktopHeader__navItem___cn2G styles_DesktopHeader__navItemRight__bFluc" aria-label="Toggle CONTACT US subnav dropdown"><a href="/contact"><button tabindex="0" role="button" aria-label="CONTACT US" class="styles_DesktopHeader__button__ZPeHZ styles_Button__nS4qV styles_Button__textCenter__odBAB styles_Button__Small__hoMkt styles_Button__GrayTextWhiteBg__9tz0E styles_Button--centered__mbVdz styles_Button--uppercase__dB7jF">CONTACT US </button></a></li><li class="styles_DesktopHeader__navItem___cn2G" aria-label="Toggle START FREE subnav dropdown"><a href="/try-redpanda"><button tabindex="0" role="button" aria-label="START FREE" class="styles_DesktopHeader__button__ZPeHZ styles_Button__nS4qV styles_Button__textCenter__odBAB styles_Button__Small__hoMkt styles_Button__WhiteTextRedBg__Lfo3_ styles_Button--centered__mbVdz styles_Button--uppercase__dB7jF">START FREE </button></a></li></ul></nav></header><header role="contentinfo" class="styles_MobileHeader__xZ_BX"><div class="styles_MobileHeader__wrapper__4L1iQ"><a aria-label="Red Panda Logo" href="/"><img src="https://images.ctfassets.net/paqvtpyf8rwu/5wODDUzLP7teOGDS7M5m39/466f646e44c2ccc4e78236b17d7d68ac/Logo.svg" alt="" title="Redpanda Logo" width="155" height="36"/></a><button class="styles_MobileHeader__mobileNavToggle__nn5xI" role="button" tabindex="0" aria-label="Toggle mobile navigation"><svg class="styles_MobileHeader__mobileNavToggleSvg__2fqFT" xmlns="http://www.w3.org/2000/svg" viewbox="0 0 100 100"><g fill="none" fill-rule="evenodd" stroke="#344054" stroke-linecap="square" stroke-width="10"><line x1="9" y1="22" x2="91" y2="22" id="top"/><line x1="9" y1="50" x2="91" y2="50" id="middle"/><line x1="9" y1="78" x2="91" y2="78" id="bottom"/></g></svg></button></div><nav class="styles_MobileHeader__mobileNavListWrapper__72h7T"/></header></div><main class="styles_Main__1isoz"><div id="contactFormSpacer"/><div class="styles_BlogPostTemplate__iSSVr"><section aria-labelledby="Blog header section." class="styles_SectionContainer__5A7Cs" id="undefined"><div class="styles_SectionContainer__innerContainer__7yBSS styles_SectionContainer__container--relative__l9Hl_ styles_SectionContainer__yPaddingOff__K3Sje"><div class="styles_BlogPostTemplate__inlineNavigation__Uwu9T"><a class="styles_BlogPostTemplate__inlineNavigationItem__TREE5" href="/"><img src="/images/home-icon.svg" alt="Red Panda logo"/></a><img src="/images/blog-nav-arrow.svg" alt="Chevron right icon"/><a class="styles_BlogPostTemplate__inlineNavigationItem__TREE5" href="/blog">Blog</a></div><div class="styles_BlogPostTemplate__header__iCoyq"><div class="styles_BlogPostTemplate__headerCategoriesAndTimeRead__jXVd_"><a class="styles_BlogPostTemplate__headerCategory__WPYNf undefined" href="/blog/categories/tutorial"><span class="styles_BlogPostTemplate__headerCategories__GoK_M">Tutorial</span></a><a class="styles_BlogPostTemplate__headerCategory__WPYNf undefined" href="/blog/categories/ecosystem"><span class="styles_BlogPostTemplate__headerCategories__GoK_M">Ecosystem</span></a></div><h1 class="styles_RenderBreakLineText__N_T68 styles_BlogPostTemplate__headerTitle__FLqjz"/><p class="styles_BlogPostTemplate__headerDescription__jRc1e">Get insights from customer reviews using GenAI and streaming data technologies</p><div class="styles_BlogPostTemplate__headerAuthorsAndDate__1HwYR"><span>By</span><span><a class="styles_BlogPostTemplate__headerAuthor__6pTyC styles_BlogPostTemplate__headerAuthorMany__xPKC_" href="/blog/authors/dunith-dhanushka">Dunith Dhanushka</a><a class="styles_BlogPostTemplate__headerAuthor__6pTyC styles_BlogPostTemplate__headerAuthorMany__xPKC_" href="/blog/authors/mark-needham">Mark Needham</a></span><span>on</span><span>April 30, 2024</span></div><div class="styles_BlogPostTemplate__headerImageWrapper__2wo1k"><img src="https://images.ctfassets.net/paqvtpyf8rwu/35Yt5dhkzXCztvb57V2U9R/26e99bf0d7e491190b4274232345ca7e/blog-sentiment-analysis-hero.png" alt="Real-time brand sentiment analysis with ClickHouse and Redpanda"/></div><div class="styles_BlogPostSocialShare__aFil6"><div class="styles_BlogPostSocialShare__shareButtons__9UCoR"><a href="https://github.com/redpanda-data/redpanda/" target="_blank" rel="noopener noreferrer" class="styles_BlogPostSocialShare__shareButton__EQRM8"><img src="/images/share-buttons-icons/github.svg" alt="Social icon github" title="Share on github"/></a><a href="https://redpandacommunity.slack.com/join/shared_invite/zt-ng2ze1uv-l5VMWSGQHB9gp47~kNnYGA#/shared-invite/email" target="_blank" rel="noopener noreferrer" class="styles_BlogPostSocialShare__shareButton__EQRM8"><img src="/images/share-buttons-icons/slack.svg" alt="Social icon slack" title="Share on slack"/></a><a href="http://twitter.com/share?url=https%3A%2F%2Fredpanda.com%2Fblog%2Freal-time-sentiment-analysis-gen-ai&amp;text=Real-time brand sentiment analysis with ClickHouse and Redpanda by @redpandadata #redpanda #apachekafka" target="_blank" rel="noopener noreferrer" class="styles_BlogPostSocialShare__shareButton__EQRM8"><img src="/images/share-buttons-icons/twitter.svg" alt="Social icon twitter" title="Share on twitter"/></a><a href="https://www.facebook.com/sharer/sharer.php?u=https%3A%2F%2Fredpanda.com%2Fblog%2Freal-time-sentiment-analysis-gen-ai" target="_blank" rel="noopener noreferrer" class="styles_BlogPostSocialShare__shareButton__EQRM8"><img src="/images/share-buttons-icons/facebook.svg" alt="Social icon facebook" title="Share on facebook"/></a><a href="https://www.linkedin.com/shareArticle?mini=true&amp;url=https%3A%2F%2Fredpanda.com%2Fblog%2Freal-time-sentiment-analysis-gen-ai" target="_blank" rel="noopener noreferrer" class="styles_BlogPostSocialShare__shareButton__EQRM8"><img src="/images/share-buttons-icons/linkedin.svg" alt="Social icon linkedin" title="Share on linkedin"/></a></div><div class="styles_BlogPostSocialShare__shareEmailAndCopy__rrpb3"><a href="mailto:?subject=Real-time brand sentiment analysis with ClickHouse and Redpanda&amp;body=https%3A%2F%2Fredpanda.com%2Fblog%2Freal-time-sentiment-analysis-gen-ai" class="styles_BlogPostSocialShare__shareButton__EQRM8"><img src="/images/share-buttons-icons/email.svg" alt="email icon" title="Share by email"/></a><button class="styles_BlogPostSocialShare__shareCopy__yXhMP styles_BlogPostSocialShare__shareButton__EQRM8" role="button" aria-label="Copy link to clipboard"><img src="/images/share-buttons-icons/copy.svg" alt="copy icon" title="Copy link"/><p>Copy link</p></button></div></div></div></div></section><section aria-labelledby="Blog post content section." class="styles_SectionContainer__5A7Cs styles_BlogPostTemplate__content__wrapper__qQYwt" id="undefined"><div class="styles_SectionContainer__innerContainer__7yBSS styles_SectionContainer__container--relative__l9Hl_ styles_SectionContainer__yPaddingOff__K3Sje"><div class="styles_BlogPostTemplate__content__with_sidebar__1sJ0_"><div class="col-span-2"><div class="styles_BlogRichText__zXy83"><p>Collecting, processing, and reacting to customer reviews in real time is crucial for brands. Firstly, it allows them to immediately identify and address issues for improved customer satisfaction. Secondly, it provides them with continuous feedback, which can be used to improve products or services. Lastly, it enables them to stay on top of changing customer preferences and market trends, giving them a competitive edge.</p><p>In this post, we’ll build a brand sentiment analysis solution by combining several streaming data and Generative AI technologies, including <a href="https://redpanda.com/what-is-redpanda">Redpanda</a>, <a href="https://clickhouse.com/">ClickHouse</a>, and OpenAI.</p><p/><h2 id="the-use-case:-customer-sentiment-analysis-for-a-cat-food-brand">The use case: customer sentiment analysis for a cat food brand</h2><p>Imagine a cat food brand creatively called “MeowMunch.” Customers engage with the brand across several touchpoints, including the website, mobile app, social media, and third party review platforms to publish product reviews.</p><p>Ultimately, the solution will collect these reviews into Redpanda. For this tutorial we’ll use a Python script to simulate a review stream and publish it to a Redpanda topic. Before writing to Redpanda, this script will also use an OpenAI embedding model to generate vector embeddings for the review texts.</p><p>Next, we’ll configure the ClickHouse server to ingest reviews from Redpanda and make them ready for querying. Finally, the brand marketing team is presented with a web frontend — a Streamlit application that allows them to understand customer sentiment by asking questions like:</p><ul><li><p>What are the most frequent words among reviews?</p></li><li><p>What are the reviews with negative sentiment?</p></li><li><p>What reviews say that the product is too expensive?</p></li></ul><p>To give you a better idea, here’s a brief overview of the architecture. </p><figure class="styles_FigureCaption__2hP9d"><img class="styles_FigureCaption__img__COw2_" src="https://images.ctfassets.net/paqvtpyf8rwu/2s8HQoqPV4MUdnGGM3MoiH/05db301c66b64c7ad12191946a627258/blog-sentiment-analysis-img1.png" alt="Architecture of the brand sentiment analysis solution "/><figcaption class="styles_FigureCaption__figcaption__Loym0">Architecture of the brand sentiment analysis solution </figcaption></figure><p>Before we dive into the step-by-step process of building this solution, here’s what you’ll need to get started.</p><p/><h2 id="prerequisites">Prerequisites</h2><p>For this tutorial, you’ll need a few things installed on your local machine.</p><p/><h3>1. Clone the GitHub repo</h3><p>The completed source code for the tutorial is available at the following GitHub repository. Go ahead and clone it to your local workstation.</p><div class="styles_CodeSnippet__wrapper__BU2Ol"><pre style="text-align:left;white-space:pre;word-spacing:normal;word-break:break-all;word-wrap:normal;color:#eee;background:#2f2f2f;font-family:Roboto Mono, monospace;font-size:1em;line-height:1.5em;-moz-tab-size:4;-o-tab-size:4;tab-size:4;-webkit-hyphens:none;-moz-hyphens:none;-ms-hyphens:none;hyphens:none;overflow:auto;position:relative;margin:0.5em 0;padding:1rem;width:100%"><code class="language-javascript" style="text-align:left;white-space:pre;word-spacing:normal;word-break:normal;word-wrap:normal;color:#eee;background:#2f2f2f;font-family:Roboto Mono, monospace;font-size:1em;line-height:1.5em;-moz-tab-size:4;-o-tab-size:4;tab-size:4;-webkit-hyphens:none;-moz-hyphens:none;-ms-hyphens:none;hyphens:none"><span>git clone https</span><span class="token token" style="color:#89ddff">:</span><span class="token token" style="color:#89ddff">/</span><span class="token token" style="color:#89ddff">/</span><span>github</span><span class="token token" style="color:#89ddff">.</span><span class="token token property-access">com</span><span class="token token" style="color:#89ddff">/</span><span>dunithd</span><span class="token token" style="color:#89ddff">/</span><span>redpanda</span><span class="token token" style="color:#89ddff">-</span><span>clickhouse</span><span class="token token" style="color:#89ddff">-</span><span>vector</span><span class="token token" style="color:#89ddff">-</span><span>search</span><span class="token token" style="color:#89ddff">.</span><span class="token token property-access">git</span><span> </span><span>cd redpanda</span><span class="token token" style="color:#89ddff">-</span><span>clickhouse</span><span class="token token" style="color:#89ddff">-</span><span>vector</span><span class="token token" style="color:#89ddff">-</span><span>search</span><span class="token token" style="color:#89ddff">/</span></code></pre></div><p/><h3>2. Install Python dependencies</h3><p>Before proceeding with the tutorial, ensure you have Python 3.x installed locally. You should also set up a Python virtual environment, like <code>venv</code>, to handle the dependencies.</p><p>Creating a virtual environment provides a sandbox for your project, preventing dependencies and versions from clashing with those of other projects. The command below creates and activates a new Python virtual environment named “vector-search”.</p><div class="styles_CodeSnippet__wrapper__BU2Ol"><pre style="text-align:left;white-space:pre;word-spacing:normal;word-break:break-all;word-wrap:normal;color:#eee;background:#2f2f2f;font-family:Roboto Mono, monospace;font-size:1em;line-height:1.5em;-moz-tab-size:4;-o-tab-size:4;tab-size:4;-webkit-hyphens:none;-moz-hyphens:none;-ms-hyphens:none;hyphens:none;overflow:auto;position:relative;margin:0.5em 0;padding:1rem;width:100%"><code class="language-javascript" style="text-align:left;white-space:pre;word-spacing:normal;word-break:normal;word-wrap:normal;color:#eee;background:#2f2f2f;font-family:Roboto Mono, monospace;font-size:1em;line-height:1.5em;-moz-tab-size:4;-o-tab-size:4;tab-size:4;-webkit-hyphens:none;-moz-hyphens:none;-ms-hyphens:none;hyphens:none"><span>python3 </span><span class="token token" style="color:#89ddff">-</span><span>m venv vector</span><span class="token token" style="color:#89ddff">-</span><span>search </span><span>source vector</span><span class="token token" style="color:#89ddff">-</span><span>search</span><span class="token token" style="color:#89ddff">/</span><span>bin</span><span class="token token" style="color:#89ddff">/</span><span>activate</span></code></pre></div><p>Now run the following command to install the Python dependencies for OpenAI, ClickHouse Connect, Kafka, and Streamlit.</p><div class="styles_CodeSnippet__wrapper__BU2Ol"><pre style="text-align:left;white-space:pre;word-spacing:normal;word-break:break-all;word-wrap:normal;color:#eee;background:#2f2f2f;font-family:Roboto Mono, monospace;font-size:1em;line-height:1.5em;-moz-tab-size:4;-o-tab-size:4;tab-size:4;-webkit-hyphens:none;-moz-hyphens:none;-ms-hyphens:none;hyphens:none;overflow:auto;position:relative;margin:0.5em 0;padding:1rem;width:100%"><code class="language-javascript" style="text-align:left;white-space:pre;word-spacing:normal;word-break:normal;word-wrap:normal;color:#eee;background:#2f2f2f;font-family:Roboto Mono, monospace;font-size:1em;line-height:1.5em;-moz-tab-size:4;-o-tab-size:4;tab-size:4;-webkit-hyphens:none;-moz-hyphens:none;-ms-hyphens:none;hyphens:none"><span>pip install </span><span class="token token" style="color:#89ddff">-</span><span>r requirements</span><span class="token token" style="color:#89ddff">.</span><span class="token token property-access">txt</span></code></pre></div><p/><h3>3. Obtain an OpenAI API key</h3><p>This tutorial will invoke several OpenAI APIs for creating vector embeddings. If you haven't already, create an account with the <a href="https://platform.openai.com/overview">OpenAI Platform</a> to obtain an API key. Keep the key safe for later.</p><figure class="styles_FigureCaption__2hP9d"><img class="styles_FigureCaption__img__COw2_" src="https://images.ctfassets.net/paqvtpyf8rwu/6pPp52Vx8rPJbSSe2wxpUI/6c28188515567dd2caf4f27e0006caff/blog-sentiment-analysis-img2.png" alt="Creating an OpenAI API key"/><figcaption class="styles_FigureCaption__figcaption__Loym0">Creating an OpenAI API key</figcaption></figure><p/><h3>4. Create and configure a Redpanda cluster</h3><p>Let’s get fancy with this tutorial and use a serverless Redpanda cluster. You can create up to five serverless Redpanda clusters with unlimited storage, so sign up for a <a href="https://redpanda.com/try-redpanda/cloud-trial#serverless">free trial</a> and follow the on-screen instructions to create a Redpanda cluster in your preferred cloud region.</p><p>Once your cluster is ready, access the <b>Topics</b> menu in the left sidebar to create a topic called <code>reviews</code>. Leave the values on the UI as defaults. We’ll use this topic to ingest product reviews.</p><p>Next, you’ll need a user account to connect to this cluster. Click on the <b>Security</b> menu on the left sidebar to open the Kafka Access Control page. Then, create a new user account by clicking the <b>Create user</b> button.</p><p>Let’s call it <code>demouser</code>. Remember the generated password for the account.</p><figure class="styles_FigureCaption__2hP9d"><img class="styles_FigureCaption__img__COw2_" src="https://images.ctfassets.net/paqvtpyf8rwu/16wckYEbDZhliOZJ4H0O22/3da8a50b205ddd9804661b44f8a08d62/blog-sentiment-analysis-img3.png" alt="Creating and configuring a Redpanda cluster"/><figcaption class="styles_FigureCaption__figcaption__Loym0">Creating and configuring a Redpanda cluster</figcaption></figure><p>Finally, select the user account, and click on the <b>Allow all operations</b> to grant <code>demouser</code> account all read/write permissions. You can fine-tune the ACLs as needed.</p><figure class="styles_FigureCaption__2hP9d"><img class="styles_FigureCaption__img__COw2_" src="https://images.ctfassets.net/paqvtpyf8rwu/dShVtMRToFMx2PeBs6pgz/c3f7693f9b8938bc92e78db026a35064/blog-sentiment-analysis-img4.png" alt="Configuring the read/write permissions"/><figcaption class="styles_FigureCaption__figcaption__Loym0">Configuring the read/write permissions</figcaption></figure><p/><h3>5. Hydrate the Redpanda topic</h3><p>Next, let’s hydrate the <code>reviews</code> topic with some product reviews. The <code>./data/reviews.json</code> file contains 20 reviews about the MeowMunch product. A single review looks like this:</p><div class="styles_CodeSnippet__wrapper__BU2Ol"><pre style="text-align:left;white-space:pre;word-spacing:normal;word-break:break-all;word-wrap:normal;color:#eee;background:#2f2f2f;font-family:Roboto Mono, monospace;font-size:1em;line-height:1.5em;-moz-tab-size:4;-o-tab-size:4;tab-size:4;-webkit-hyphens:none;-moz-hyphens:none;-ms-hyphens:none;hyphens:none;overflow:auto;position:relative;margin:0.5em 0;padding:1rem;width:100%"><code class="language-javascript" style="text-align:left;white-space:pre;word-spacing:normal;word-break:normal;word-wrap:normal;color:#eee;background:#2f2f2f;font-family:Roboto Mono, monospace;font-size:1em;line-height:1.5em;-moz-tab-size:4;-o-tab-size:4;tab-size:4;-webkit-hyphens:none;-moz-hyphens:none;-ms-hyphens:none;hyphens:none"><span class="token token" style="color:#89ddff">{</span><span> </span><span> </span><span class="token token string-property" style="color:#80cbc4">"user"</span><span class="token token" style="color:#89ddff">:</span><span> </span><span class="token token" style="color:#a5e844">"Tracy"</span><span class="token token" style="color:#89ddff">,</span><span> </span><span> </span><span class="token token string-property" style="color:#80cbc4">"review"</span><span class="token token" style="color:#89ddff">:</span><span> </span><span class="token token" style="color:#a5e844">"The Kitten Li'l Nibbles are great for my kitten, but they're a bit expensive for the quantity provided."</span><span> </span><span/><span class="token token" style="color:#89ddff">}</span></code></pre></div><p>We'll publish these JSON objects into the <code>reviews</code> Redpanda topic by running the <code>reviews_producer.py</code> script. This Python script carries out several important tasks.</p><p>First, it reads and parses the <code>./data/reviews.json</code> file into an array. This array is then processed to create vector embeddings for each review object using the OpenAI model <code>text-embedding-3-small</code>. At this point, you’ll need the OpenAI API key.</p><div class="styles_CodeSnippet__wrapper__BU2Ol"><pre style="text-align:left;white-space:pre;word-spacing:normal;word-break:break-all;word-wrap:normal;color:#eee;background:#2f2f2f;font-family:Roboto Mono, monospace;font-size:1em;line-height:1.5em;-moz-tab-size:4;-o-tab-size:4;tab-size:4;-webkit-hyphens:none;-moz-hyphens:none;-ms-hyphens:none;hyphens:none;overflow:auto;position:relative;margin:0.5em 0;padding:1rem;width:100%"><code class="language-javascript" style="text-align:left;white-space:pre;word-spacing:normal;word-break:normal;word-wrap:normal;color:#eee;background:#2f2f2f;font-family:Roboto Mono, monospace;font-size:1em;line-height:1.5em;-moz-tab-size:4;-o-tab-size:4;tab-size:4;-webkit-hyphens:none;-moz-hyphens:none;-ms-hyphens:none;hyphens:none"><span>def </span><span class="token token" style="color:#c792ea">get_embedding</span><span class="token token" style="color:#89ddff">(</span><span>text</span><span class="token token" style="color:#89ddff">,</span><span> model</span><span class="token token" style="color:#89ddff">=</span><span class="token token" style="color:#a5e844">"text-embedding-3-small"</span><span class="token token" style="color:#89ddff">)</span><span class="token token" style="color:#89ddff">:</span><span> </span><span> text </span><span class="token token" style="color:#89ddff">=</span><span> text</span><span class="token token" style="color:#89ddff">.</span><span class="token token method property-access" style="color:#c792ea">replace</span><span class="token token" style="color:#89ddff">(</span><span class="token token" style="color:#a5e844">"\\n"</span><span class="token token" style="color:#89ddff">,</span><span> </span><span class="token token" style="color:#a5e844">" "</span><span class="token token" style="color:#89ddff">)</span><span> </span><span> </span><span class="token token control-flow" style="color:#c792ea">return</span><span> client</span><span class="token token" style="color:#89ddff">.</span><span class="token token property-access">embeddings</span><span class="token token" style="color:#89ddff">.</span><span class="token token method property-access" style="color:#c792ea">create</span><span class="token token" style="color:#89ddff">(</span><span>input </span><span class="token token" style="color:#89ddff">=</span><span> </span><span class="token token" style="color:#89ddff">[</span><span>text</span><span class="token token" style="color:#89ddff">]</span><span class="token token" style="color:#89ddff">,</span><span> model</span><span class="token token" style="color:#89ddff">=</span><span>model</span><span class="token token" style="color:#89ddff">)</span><span class="token token" style="color:#89ddff">.</span><span class="token token property-access">data</span><span class="token token" style="color:#89ddff">[</span><span class="token token" style="color:#fd9170">0</span><span class="token token" style="color:#89ddff">]</span><span class="token token" style="color:#89ddff">.</span><span class="token token property-access">embedding</span></code></pre></div><p>Each review object is enriched with the generated vector embedding and finally published to the <code>reviews</code> Redpanda topic.</p><div class="styles_CodeSnippet__wrapper__BU2Ol"><pre style="text-align:left;white-space:pre;word-spacing:normal;word-break:break-all;word-wrap:normal;color:#eee;background:#2f2f2f;font-family:Roboto Mono, monospace;font-size:1em;line-height:1.5em;-moz-tab-size:4;-o-tab-size:4;tab-size:4;-webkit-hyphens:none;-moz-hyphens:none;-ms-hyphens:none;hyphens:none;overflow:auto;position:relative;margin:0.5em 0;padding:1rem;width:100%"><code class="language-javascript" style="text-align:left;white-space:pre;word-spacing:normal;word-break:normal;word-wrap:normal;color:#eee;background:#2f2f2f;font-family:Roboto Mono, monospace;font-size:1em;line-height:1.5em;-moz-tab-size:4;-o-tab-size:4;tab-size:4;-webkit-hyphens:none;-moz-hyphens:none;-ms-hyphens:none;hyphens:none"><span class="token token control-flow" style="color:#c792ea">for</span><span> review </span><span class="token token" style="color:#c792ea">in</span><span> </span><span class="token token literal-property" style="color:#80cbc4">reviews</span><span class="token token" style="color:#89ddff">:</span><span> </span><span> review</span><span class="token token" style="color:#89ddff">[</span><span class="token token" style="color:#a5e844">'embedding'</span><span class="token token" style="color:#89ddff">]</span><span> </span><span class="token token" style="color:#89ddff">=</span><span> </span><span class="token token" style="color:#c792ea">get_embedding</span><span class="token token" style="color:#89ddff">(</span><span>review</span><span class="token token" style="color:#89ddff">[</span><span class="token token" style="color:#a5e844">'review'</span><span class="token token" style="color:#89ddff">]</span><span class="token token" style="color:#89ddff">)</span><span> </span><span> producer</span><span class="token token" style="color:#89ddff">.</span><span class="token token method property-access" style="color:#c792ea">send</span><span class="token token" style="color:#89ddff">(</span><span>topic</span><span class="token token" style="color:#89ddff">,</span><span> value</span><span class="token token" style="color:#89ddff">=</span><span>review</span><span class="token token" style="color:#89ddff">)</span></code></pre></div><p>Before running the script, set the environment variables required to connect the Redpanda cluster you created above. The variables are self-explanatory and you can find the relevant values in the <b>Overview</b> section of the cluster.</p><div class="styles_CodeSnippet__wrapper__BU2Ol"><pre style="text-align:left;white-space:pre;word-spacing:normal;word-break:break-all;word-wrap:normal;color:#eee;background:#2f2f2f;font-family:Roboto Mono, monospace;font-size:1em;line-height:1.5em;-moz-tab-size:4;-o-tab-size:4;tab-size:4;-webkit-hyphens:none;-moz-hyphens:none;-ms-hyphens:none;hyphens:none;overflow:auto;position:relative;margin:0.5em 0;padding:1rem;width:100%"><code class="language-javascript" style="text-align:left;white-space:pre;word-spacing:normal;word-break:normal;word-wrap:normal;color:#eee;background:#2f2f2f;font-family:Roboto Mono, monospace;font-size:1em;line-height:1.5em;-moz-tab-size:4;-o-tab-size:4;tab-size:4;-webkit-hyphens:none;-moz-hyphens:none;-ms-hyphens:none;hyphens:none"><span class="token token module" style="color:#c792ea">export</span><span> </span><span class="token token" style="color:#c792ea">RP_HOST</span><span class="token token" style="color:#89ddff">=</span><span class="token token" style="color:#a5e844">"&lt;redpanda_bootrap_server&gt;:9092"</span><span> </span><span/><span class="token token module" style="color:#c792ea">export</span><span> </span><span class="token token" style="color:#c792ea">RP_USER</span><span class="token token" style="color:#89ddff">=</span><span class="token token" style="color:#a5e844">"demouser"</span><span> </span><span/><span class="token token module" style="color:#c792ea">export</span><span> </span><span class="token token" style="color:#c792ea">RP_PASSWORD</span><span class="token token" style="color:#89ddff">=</span><span class="token token" style="color:#a5e844">"&lt;put_your_generated_password_here&gt;"</span><span> </span><span/><span class="token token module" style="color:#c792ea">export</span><span> </span><span class="token token" style="color:#c792ea">OPENAI_API_KEY</span><span class="token token" style="color:#89ddff">=</span><span class="token token" style="color:#a5e844">"&lt;put_your_api_key_here&gt;"</span></code></pre></div><p>Finally, run the script to hydrate the topic.</p><div class="styles_CodeSnippet__wrapper__BU2Ol"><pre style="text-align:left;white-space:pre;word-spacing:normal;word-break:break-all;word-wrap:normal;color:#eee;background:#2f2f2f;font-family:Roboto Mono, monospace;font-size:1em;line-height:1.5em;-moz-tab-size:4;-o-tab-size:4;tab-size:4;-webkit-hyphens:none;-moz-hyphens:none;-ms-hyphens:none;hyphens:none;overflow:auto;position:relative;margin:0.5em 0;padding:1rem;width:100%"><code class="language-javascript" style="text-align:left;white-space:pre;word-spacing:normal;word-break:normal;word-wrap:normal;color:#eee;background:#2f2f2f;font-family:Roboto Mono, monospace;font-size:1em;line-height:1.5em;-moz-tab-size:4;-o-tab-size:4;tab-size:4;-webkit-hyphens:none;-moz-hyphens:none;-ms-hyphens:none;hyphens:none"><span>python3 </span><span class="token token" style="color:#89ddff">.</span><span class="token token" style="color:#89ddff">/</span><span>reviews_producer</span><span class="token token" style="color:#89ddff">.</span><span class="token token property-access">py</span></code></pre></div><p>You should see the <code>reviews</code> topic is now populated with reviews.</p><figure class="styles_FigureCaption__2hP9d"><img class="styles_FigureCaption__img__COw2_" src="https://images.ctfassets.net/paqvtpyf8rwu/11VrFjcuFaWDvil5ZyfSdA/cb389ce61a8d213c505961ba86f419ea/blog-sentiment-analysis-img5.png" alt="Reviews in the Redpanda “reviews” topic"/><figcaption class="styles_FigureCaption__figcaption__Loym0">Reviews in the Redpanda “reviews” topic</figcaption></figure><p/><h3>6. Ingest reviews into ClickHouse</h3><p>Great! We now have reviews in Redpanda. Next, we’ll configure ClickHouse to ingest them from Redpanda and prepare them for semantic search queries. We’ll use <a href="https://clickhouse.com/cloud">ClickHouse Cloud</a>, which is the cloud-hosted and fully managed version of the ClickHouse database. </p><p>Create an account with ClickHouse Cloud and sign into the web console. Then, create a <a href="https://clickhouse.com/cloud/clickpipes">ClickPipe</a> to ingest from the Redpanda cluster you just created. </p><figure class="styles_FigureCaption__2hP9d"><img class="styles_FigureCaption__img__COw2_" src="https://images.ctfassets.net/paqvtpyf8rwu/2b3JjBAEmcqNoahyN9Ugk1/7748b1baa43914765b089bff15a91511/blog-sentiment-analysis-img6.png" alt="Creating a ClickPipe in the Clickhouse web console"/><figcaption class="styles_FigureCaption__figcaption__Loym0">Creating a ClickPipe in the Clickhouse web console</figcaption></figure><p>Provide the Redpanda cluster credentials when prompted, then select the <code>reviews</code> topic as the data source, and define the target table, <code>reviews</code>, as follows.</p><figure class="styles_FigureCaption__2hP9d"><img class="styles_FigureCaption__img__COw2_" src="https://images.ctfassets.net/paqvtpyf8rwu/5468qEohPjrZrLDMUyuxO3/ea971f31b17ea6d22a82e846db33f77d/blog-sentiment-analysis-img7.png" alt="Configuring the ClickPipe in the Clickhouse web console"/><figcaption class="styles_FigureCaption__figcaption__Loym0">Configuring the ClickPipe in the Clickhouse web console</figcaption></figure><p>Once configured, this ClickPipe will commence the ingestion in seconds. </p><figure class="styles_FigureCaption__2hP9d"><img class="styles_FigureCaption__img__COw2_" src="https://images.ctfassets.net/paqvtpyf8rwu/6ZzE0WjmFDLObyXxck2mK0/a0ad69bd736f2736d1fc6cdccb9b88cb/blog-sentiment-analysis-img8.png" alt="Ingesting data into ClickPipe from the Redpanda topic"/><figcaption class="styles_FigureCaption__figcaption__Loym0">Ingesting data into ClickPipe from the Redpanda topic</figcaption></figure><p>Click on the <b>SQL Console</b> menu on the left sidebar to check the reviews table is now populated with reviews.</p><figure class="styles_FigureCaption__2hP9d"><img class="styles_FigureCaption__img__COw2_" src="https://images.ctfassets.net/paqvtpyf8rwu/17Fg6Y1M2l5RdlTR0ADMYJ/762f5979256ea60efaa34114a4f75395/blog-sentiment-analysis-img9.png" alt="Reviews populating the table in the SQL Console"/><figcaption class="styles_FigureCaption__figcaption__Loym0">Reviews populating the table in the SQL Console</figcaption></figure><p>For a more detailed walkthrough, watch <a href="https://www.youtube.com/watch?v=Lg1jDRy0OlE">this video on setting up a ClickPipe to Redpanda</a>.</p><p/><h3>7. Run the Streamlit frontend</h3><p>Now we have the necessary data infrastructure. But, not everyone in the company knows how to work with SQL. If we’re going to make this data accessible to other users, we need a more user-friendly way for them to access the data.</p><p><a href="https://streamlit.io/">Streamlit</a> is popular for quickly building prototypes of interactive frontends. So, we’re going to create an app that’ll let the user see common phrases used in reviews, find reviews similar to a provided search term, and see reviews similar to each other. </p><p>This dashboard would be mighty useful to MeowMunch's marketing team!</p><p>Start the Streamlit dashboard by running:</p><div class="styles_CodeSnippet__wrapper__BU2Ol"><pre style="text-align:left;white-space:pre;word-spacing:normal;word-break:break-all;word-wrap:normal;color:#eee;background:#2f2f2f;font-family:Roboto Mono, monospace;font-size:1em;line-height:1.5em;-moz-tab-size:4;-o-tab-size:4;tab-size:4;-webkit-hyphens:none;-moz-hyphens:none;-ms-hyphens:none;hyphens:none;overflow:auto;position:relative;margin:0.5em 0;padding:1rem;width:100%"><code class="language-javascript" style="text-align:left;white-space:pre;word-spacing:normal;word-break:normal;word-wrap:normal;color:#eee;background:#2f2f2f;font-family:Roboto Mono, monospace;font-size:1em;line-height:1.5em;-moz-tab-size:4;-o-tab-size:4;tab-size:4;-webkit-hyphens:none;-moz-hyphens:none;-ms-hyphens:none;hyphens:none"><span>streamlit run app</span><span class="token token" style="color:#89ddff">.</span><span class="token token property-access">py</span></code></pre></div><p>The dashboard will be available at <a href="http://localhost:8501/">http://localhost:8501</a>. Next, let’s break down each widget displayed in the dashboard. The first widget aggregates the most popular phrases across all the reviews of MeowMunch. Here’s what it would look like:</p><figure class="styles_FigureCaption__2hP9d"><img class="styles_FigureCaption__img__COw2_" src="https://images.ctfassets.net/paqvtpyf8rwu/4bSTHLKfQANLqqf9kTMPQj/05b9dfd92e783dc577b752d2d756ddaa/blog-sentiment-analysis-img10.png" alt="Widget of popular phrases used in customer reviews"/><figcaption class="styles_FigureCaption__figcaption__Loym0">Widget of popular phrases used in customer reviews</figcaption></figure><p>We did this using the following query, which computes n-grams of three tokens, excluding n-grams that start or end with common stop words.</p><div class="styles_CodeSnippet__wrapper__BU2Ol"><pre style="text-align:left;white-space:pre;word-spacing:normal;word-break:break-all;word-wrap:normal;color:#eee;background:#2f2f2f;font-family:Roboto Mono, monospace;font-size:1em;line-height:1.5em;-moz-tab-size:4;-o-tab-size:4;tab-size:4;-webkit-hyphens:none;-moz-hyphens:none;-ms-hyphens:none;hyphens:none;overflow:auto;position:relative;margin:0.5em 0;padding:1rem;width:100%"><code class="language-javascript" style="text-align:left;white-space:pre;word-spacing:normal;word-break:normal;word-wrap:normal;color:#eee;background:#2f2f2f;font-family:Roboto Mono, monospace;font-size:1em;line-height:1.5em;-moz-tab-size:4;-o-tab-size:4;tab-size:4;-webkit-hyphens:none;-moz-hyphens:none;-ms-hyphens:none;hyphens:none"><span class="token token" style="color:#c792ea">WITH</span><span> reviewTokens </span><span class="token token" style="color:#c792ea">AS</span><span> </span><span> </span><span class="token token" style="color:#c792ea">SELECT</span><span> </span><span class="token token" style="color:#c792ea">tokens</span><span class="token token" style="color:#89ddff">(</span><span>review</span><span class="token token" style="color:#89ddff">)</span><span> </span><span class="token token" style="color:#c792ea">AS</span><span> textTokens </span><span> </span><span class="token token" style="color:#c792ea">FROM</span><span> reviews </span><span/><span class="token token" style="color:#c792ea">SELECT</span><span> </span><span> </span><span class="token token" style="color:#c792ea">arrayJoin</span><span class="token token" style="color:#89ddff">(</span><span class="token token control-flow" style="color:#c792ea">if</span><span class="token token" style="color:#89ddff">(</span><span class="token token" style="color:#c792ea">length</span><span class="token token" style="color:#89ddff">(</span><span>textTokens</span><span class="token token" style="color:#89ddff">)</span><span> </span><span class="token token" style="color:#89ddff">&lt;</span><span> </span><span class="token token" style="color:#fd9170">3</span><span class="token token" style="color:#89ddff">,</span><span> </span><span class="token token" style="color:#89ddff">[</span><span>textTokens</span><span class="token token" style="color:#89ddff">]</span><span class="token token" style="color:#89ddff">,</span><span> </span><span class="token token" style="color:#c792ea">arrayShingles</span><span class="token token" style="color:#89ddff">(</span><span>textTokens</span><span class="token token" style="color:#89ddff">,</span><span> </span><span class="token token" style="color:#fd9170">3</span><span class="token token" style="color:#89ddff">)</span><span class="token token" style="color:#89ddff">)</span><span class="token token" style="color:#89ddff">)</span><span> </span><span class="token token" style="color:#c792ea">AS</span><span> shingle</span><span class="token token" style="color:#89ddff">,</span><span> </span><span> </span><span class="token token" style="color:#c792ea">count</span><span class="token token" style="color:#89ddff">(</span><span class="token token" style="color:#89ddff">)</span><span> </span><span/><span class="token token" style="color:#c792ea">FROM</span><span> reviewTokens </span><span/><span class="token token" style="color:#c792ea">WHERE</span><span> </span><span class="token token" style="color:#c792ea">numberOfStopWords</span><span class="token token" style="color:#89ddff">(</span><span>shingle</span><span class="token token" style="color:#89ddff">)</span><span> </span><span class="token token" style="color:#89ddff">&lt;=</span><span> </span><span class="token token" style="color:#fd9170">1</span><span> </span><span class="token token" style="color:#c792ea">AND</span><span> not </span><span class="token token" style="color:#c792ea">startsOrEndsWithStopWord</span><span class="token token" style="color:#89ddff">(</span><span>shingle</span><span class="token token" style="color:#89ddff">)</span><span> </span><span/><span class="token token" style="color:#c792ea">GROUP</span><span> </span><span class="token token" style="color:#c792ea">BY</span><span> </span><span class="token token" style="color:#c792ea">ALL</span><span> </span><span/><span class="token token" style="color:#c792ea">ORDER</span><span> </span><span class="token token" style="color:#c792ea">BY</span><span> </span><span class="token token" style="color:#c792ea">count</span><span class="token token" style="color:#89ddff">(</span><span class="token token" style="color:#89ddff">)</span><span> </span><span class="token token" style="color:#c792ea">DESC</span><span> </span><span/><span class="token token" style="color:#c792ea">LIMIT</span><span> </span><span class="token token" style="color:#fd9170">10</span></code></pre></div><p/><h3>8. Find similar reviews</h3><p>Next we have the “similar reviews” widget. This widget allows you to search for similar reviews for a selected view. </p><figure class="styles_FigureCaption__2hP9d"><img class="styles_FigureCaption__img__COw2_" src="https://images.ctfassets.net/paqvtpyf8rwu/3w8yMYgw1YuMrzjEdJDe6m/6960e1cd996aa8a8b51397e8c32e6ad6/blog-sentiment-analysis-img11.png" alt="Widget showing reviews with similar phrases"/><figcaption class="styles_FigureCaption__figcaption__Loym0">Widget showing reviews with similar phrases</figcaption></figure><p>This one uses vector search in ClickHouse. We take the embedding vector for the selected review, compute the <code>cosineDistance</code> from that review to all the other ones, and then show the ones with the smallest score. The first result is always ourselves and then the others have semantically similar text. The query to do that is shown below.</p><div class="styles_CodeSnippet__wrapper__BU2Ol"><pre style="text-align:left;white-space:pre;word-spacing:normal;word-break:break-all;word-wrap:normal;color:#eee;background:#2f2f2f;font-family:Roboto Mono, monospace;font-size:1em;line-height:1.5em;-moz-tab-size:4;-o-tab-size:4;tab-size:4;-webkit-hyphens:none;-moz-hyphens:none;-ms-hyphens:none;hyphens:none;overflow:auto;position:relative;margin:0.5em 0;padding:1rem;width:100%"><code class="language-javascript" style="text-align:left;white-space:pre;word-spacing:normal;word-break:normal;word-wrap:normal;color:#eee;background:#2f2f2f;font-family:Roboto Mono, monospace;font-size:1em;line-height:1.5em;-moz-tab-size:4;-o-tab-size:4;tab-size:4;-webkit-hyphens:none;-moz-hyphens:none;-ms-hyphens:none;hyphens:none"><span class="token token" style="color:#89ddff">--</span><span> </span><span class="token token" style="color:#c792ea">A</span><span> </span><span class="token token" style="color:#c792ea">function</span><span> that gets the embedding vector </span><span class="token token control-flow" style="color:#c792ea">for</span><span> the selected review </span><span/><span class="token token" style="color:#c792ea">CREATE</span><span> </span><span class="token token" style="color:#c792ea">OR</span><span> </span><span class="token token" style="color:#c792ea">REPLACE</span><span> </span><span class="token token" style="color:#c792ea">FUNCTION</span><span> getEmbedding </span><span class="token token" style="color:#c792ea">AS</span><span> </span><span class="token token" style="color:#89ddff">(</span><span>selectedReview</span><span class="token token" style="color:#89ddff">)</span><span> </span><span class="token token" style="color:#89ddff">-</span><span class="token token" style="color:#89ddff">&gt;</span><span> </span><span class="token token" style="color:#89ddff">(</span><span> </span> select embedding <span> </span><span class="token token module" style="color:#c792ea">from</span><span> reviews </span><span> </span><span class="token token" style="color:#c792ea">WHERE</span><span> review </span><span class="token token" style="color:#89ddff">=</span><span> selectedReview </span><span> </span><span class="token token" style="color:#c792ea">LIMIT</span><span> </span><span class="token token" style="color:#fd9170">1</span><span> <span/><span class="token token" style="color:#c792ea">SELECT</span><span> </span><span> review</span><span class="token token" style="color:#89ddff">,</span><span>