OpenSearch Community Meeting - 2022-1122

kris · October 19, 2022, 10:07pm

OpenSearch Community Meeting - 2022-1122 - Hope we see you all there!

Agenda:

Becoming a MAINTAINER - @dblock - Principal Engineer, OpenSearch
Discuss the 2.4.0 release with @dtaivpp - Sr Dev Adv, OpenSearch

Event page
Meetup page

Meeting Link

Meeting ID: 813 0091 2180
Passcode: 503383

Would you like to present? Tag @kris @dtaivpp @nateynate and we’ll work to get you added to the agenda!

Feel free to comment on this agenda before the meeting if you want to add an item or have a question.

After the meeting, we will post the chat log and any meeting notes. We welcome you to keep the conversation going here on the forum.

========
By joining the OpenSearch Community Meeting, you grant OpenSearch, and our affiliates the right to record, film, photograph, and capture your voice and image during the OpenSearch Community Meeting (the “Recordings”). You grant to us an irrevocable, nonexclusive, perpetual, worldwide, royalty-free right and license to use, reproduce, modify, distribute, and translate, for any purpose, all or any part of the Recordings and Your Materials. For example, we may distribute Recordings or snippets of Recordings via our social media outlets.

kris · November 22, 2022, 6:55pm

Chat log from today’s meeting:

08:05:36 From Daniel Doubrovkine to Everyone:
1 minute sound meditation !
08:07:17 From Kris Freedain to Everyone:
Opensearch 2.4.0 · OpenSearch
08:09:11 From Amitai Stern to Everyone:
also your chair lost its gravity
08:09:59 From Kris Freedain to Everyone:

08:20:33 From Kris Freedain to Everyone:
Andriy! @reta
08:21:18 From David Tippett to Everyone:
Not a chance lol OpenSearch-py!
08:21:29 From Nate B. to Everyone:
opensearch-ruby! Fight!
08:21:37 From Charlotte Henkle to Everyone:
massively hard stare here, dB
08:21:38 From Andriy Redko to Everyone:
Thanks a mill guys!
08:21:48 From Minal Shah to Everyone:
Yay to Ruby!
08:22:17 From Kris Freedain to Everyone:
Thank YOU Andriy!
08:24:52 From 吴峻申 to Everyone:
thans for your sharing
08:25:00 From Charlotte Henkle to Everyone:
DANCE OFF!
08:25:37 From Charlotte Henkle to Everyone:
We would all lose to dB
08:26:30 From Amitai Stern to Everyone:
excited for 2.4!
08:27:51 From Nate B. to Everyone:
Interesting!
08:29:44 From Nate B. to Everyone:
Are we continuing the pattern of hiding experimental features behind configuration variables?
08:32:56 From Nate B. to Everyone:
OpenSearch/opensearch.release-notes-2.4.0.md at main · opensearch-project/OpenSearch · GitHub
08:32:58 From Charlotte Henkle to Everyone:
Sorry, lost the window for a moment. One of the things I realized from this release is that we haven’t been consistent on how we’re handling/exposing experimental features. Short answer is some of them are behind a config in YAML, some of them are labeled in the UI. If you’re interested in the topic (how should we handle experimental features) I’ve opened an issue to discuss it: Defining "Experimental"/"Feature Flags"/? · Issue #108 · opensearch-project/.github · GitHub
08:33:51 From Nate B. to Everyone:
The GeoJSON spec: RFC 7946: The GeoJSON Format
08:34:10 From Kris Freedain to Everyone:
Release Notes: https://github.com/opensearch-project/opensearch-build/blob/main/release-notes/opensearch-release-notes-2.4.0.md
08:34:25 From Kris Freedain to Everyone:
Release Announcement: OpenSearch 2.4.0 is available today! · OpenSearch
08:34:41 From Kris Freedain to Everyone:
Download 2.4.0: Opensearch 2.4.0 · OpenSearch
08:44:15 From Daniel Doubrovkine to Everyone:
Node roles act like labels in OpenSearch (unlike Elasticsearch) and can be dynamic (you can name them whatever you want without modifying code via config). Then they can be used as “preference”. We’re thinking that “search” nodes will get increasingly good at optimizing search if you dedicate nodes to it.
08:44:19 From Daniel Doubrovkine to Everyone:
Search back pressure is [RFC] Backpressure in Search Path · Issue #1329 · opensearch-project/OpenSearch · GitHub
08:45:15 From Daniel Doubrovkine to Everyone:
It tracks memory/cpu usage and starts rejecting and canceling running searches when threshold is breached.
08:48:47 From Nate B. to Everyone:
Kafka-esque has a new meaning after being at the museum in Prague.
08:52:47 From Kris Freedain to Everyone:
- totally does Nate
08:54:13 From Alejandro Guida to Everyone:
Sorry, I don’t have a microphone, but what is the difference of Multiple data sources and Cross Cluster Search for example?
08:57:37 From Alejandro Guida to Everyone:
Thanks
08:58:15 From Alejandro Guida to Everyone:
If in the future this is going to allow non compatible ElasticSearch/OpenSearch data sources, is going to be a really good idea
08:58:44 From Amitai Stern to Everyone:
nice idea
08:59:45 From Nate B. to Everyone:
Twitter: @OpenSearchProj
LinkedIn: linkedin.com/company/opensearch-project
Facebook: OpenSearch Project
08:59:45 From Kris Freedain to Everyone:
https://twitter.com/OpenSearchProj
08:59:56 From Kris Freedain to Everyone:
https://www.linkedin.com/company/opensearch-project/
09:00:05 From Kris Freedain to Everyone:
https://www.youtube.com/c/OpenSearchProject
09:00:11 From Kris Freedain to Everyone:
https://www.meetup.com/opensearch/
09:00:17 From Charlotte Henkle to Everyone:
I’ve got to drop…thanks! Great to see you.
09:00:21 From William Beckler, OpenSearch (he/him) to Everyone:
Awesome meeting everyone! Thank you!!!
09:00:26 From Lukáš Vlček (@Aiven.io) to Everyone:
Thanks!
09:00:28 From Alejandro Guida to Everyone:
Yes, I need to jump to another call
09:00:31 From Terry Q to Everyone:
Thanks!
09:00:32 From Kris Freedain to Everyone:
Thank you everyone!!
09:00:32 From Amitai Stern to Everyone:
thanks! bye everyone
09:00:35 From Alejandro Guida to Everyone:
Thanks
09:00:48 From Kuldeep Yadav to Everyone:
thanks

kris · November 22, 2022, 7:03pm

Today’s slides:

github.com

opensearch-project/.github/blob/main/MAINTAINERS.md#becoming-a-maintainer

- [Overview](#overview)
- [Current Maintainers](#current-maintainers)
- [Emeritus](#emeritus)

## Overview

This document contains a list of maintainers in this repo. See [opensearch-project/.github/RESPONSIBILITIES.md](https://github.com/opensearch-project/.github/blob/main/RESPONSIBILITIES.md#maintainer-responsibilities) that explains what the role of maintainer means, what maintainers do in this and other repos, and how they should be doing it. If you're interested in contributing, and becoming a maintainer, see [CONTRIBUTING](CONTRIBUTING.md).

## Current Maintainers

| Maintainer         | GitHub ID                                                 | Affiliation |
| ------------------ | --------------------------------------------------------- | ----------- |
| Charlotte Henkle   | [CEHENKLE](https://github.com/CEHENKLE)                   | Amazon      |
| Daniel Doubrovkine | [dblock](https://github.com/dblock)                       | Amazon      |
| Marc Handalian     | [mch2](https://github.com/mch2)                           | Amazon      |
| Peter Nied         | [peternied](https://github.com/peternied)                 | Amazon      |
| Sarat Vemulapalli  | [saratvemulapalli](https://github.com/saratvemulapalli)   | Amazon      |

## Emeritus

This file has been truncated. show original

github.com

opensearch-project/opensearch-build/blob/main/release-notes/opensearch-release-notes-2.4.0.md

# OpenSearch and Dashboards 2.4.0 Release Notes

## Release Highlights

The OpenSearch 2.4.0 release adds a number of new capabilities to help you enhance your search, analytics, and observability solutions. This release also marks the debut of the software’s first distribution for Windows environments, first security analytics plugin, first semantic search tools, and first solution for ingesting metric data from logs, plus new functionality to enhance cluster resiliency and protect and restore your data. Following are some highlights for this release.

### New features

* New Windows x64 distributions let you deploy OpenSearch and OpenSearch Dashboards directly in your Windows environment.
* You can now ingest metric data into OpenSearch from Prometheus or from log data aggregated within OpenSearch, enabling you to analyze and correlate data across logs, traces, and metrics.
* A new log patterns feature lets you automatically generate patterns from log data for visualization and correlation, enabling you to monitor patterns over time and identify issues.
* Point-in-time search is accessible through an API, allowing you to run different queries against a dataset that is fixed in time.
* New functionality in OpenSearch Dashboards lets you restore data from existing snapshots. Previously, restoring snapshots required using the restore API.
* New geospatial functionality in OpenSearch Dashboards includes two new field types, making it possible to index and search geographic points represented as Cartesian coordinates or as Cartesian shapes. A new GeoHex grid aggregation feature supports datasets that use the Hexagonal Hierarchical Geospatial Indexing System (H3).
* Cluster manager task throttling supports cluster resiliency by allowing you to set limits on the volume of tasks that nodes can submit to the cluster manager.
* Search backpressure supports enhanced resiliency by protecting clusters against traffic surges and other issues that can overwhelm cluster resources.
* A weighted zonal search request routing policy allows you to configure policies to route search traffic away from zones that have degraded performance or failures.
* You can now create custom filters to refine your k-NN searches. The filtering is tightly integrated with the Lucene HNSW algorithm and allows you to apply k-NN searches more efficiently with better search results and performance.

### Experimental features

This file has been truncated. show original

cuongvc1 · November 23, 2022, 9:31am

Hi Guy.
I have a problem.
i just started opensearch, i am setting up a lab for anomaly detection from log(syslog and *.log). but I don’t understand how it works yet. Why was it discovered? What is Expected Value? How calculated this value can be compared to the Feature output. and 2 Confidence . values
Does Anomaly grade have a formula for calculating no. Hope everybody help please. tks

Gsmitt · November 24, 2022, 11:40pm

Hey @cuongvc1

I think your in the wrong category, may want to try posting here.

Topic		Replies	Views
OpenSearch Community Meeting - 2023-1017 Community community-meeting	3	644	October 28, 2023
OpenSearch Community Meeting - 2022-1025 Community community-meeting	1	897	October 25, 2022
OpenSearch Community Meeting - 2022-0802 Community community-meeting	6	1158	August 5, 2022
OpenSearch Community Meeting - 2024-0227 Community community-meeting	5	324	February 28, 2024
OpenSearch Community Meeting - 2022-1220 Community community-meeting	2	959	December 20, 2022

OpenSearch Community Meeting - 2022-1122

Related topics