How to add layers to a map in OpenSearch Dashboard 1.2.0

Kibana Dashboards allows me to have layers on a map where the user can hide or show the layer depending on what they want to see.

I am struggling to understand how to do this with OpenSearch Dashboards 1.2.0. I can create a map, but there seems to be no mechanism for adding a layer that the user can hide or show while they are using our dashboard.

Does this feature exist? If so, how do I do it?

Thanks.

Hello,
The maps module as known in Kibana does not (yet) exist in OSD.
However, you can probably use the transform plugin with the help of the JS library leaflet/openstreetmap but it requires a bit of development.
However, I have also provided samples to use leaflet and layers in leaflet (see sample_vizs directory).
The plugin is also available from the Community page.
Hope it helps.
Lionel

Thank you Lionel.

I tried to install the transform plugin inside the docker container running opensearch dashboards and am getting an error:
Plugin installation was unsuccessful due to error “Plugin transformVis [1.1.0] is incompatible with OpenSearch Dashboards [1.2.0]”

Thanks for your help.

Hello,
Which release did you get ?
If using 1.2, the command line to install the plugin is $KIBANA_HOME/bin/opensearch-dashboards-plugin install https://github.com/lguillaud/osd_transform_vis/releases/download/1.2.0/transformVis-1.2.0.zip.
Please let me know.
Lionel

Thanks Lionel. I was able to install 1.2.0, and it shows up in the list of installed plugins:
alertingDashboards@1.2.0.0
anomalyDetectionDashboards@1.2.0.0
ganttChartDashboards@1.2.0.0
indexManagementDashboards@1.2.0.0
observabilityDashboards@1.2.0.0
queryWorkbenchDashboards@1.2.0.0
reportsDashboards@1.2.0.0
transformVis@1.2.0

But when I try to create a new visualization I don’t see transform.

I must have done something wrong. Thanks for the clues, I’ll keep plugging along.

Hello,
Did you add the specific configuration lines into the opensearch_dashboards.yml file ?
See related section in README file: GitHub - lguillaud/osd_transform_vis: OpenSearch-Dashboards plugin to create custom visualisations.

Extract from my Dockerfile:

...
ARG VERSION
...
# Transform viz settings
RUN echo "csp.strict: false" >> /usr/share/opensearch-dashboards/config/opensearch_dashboards.yml
RUN echo "csp.warnLegacyBrowsers: false" >> /usr/share/opensearch-dashboards/config/opensearch_dashboards.yml
RUN echo "csp.rules:" >> /usr/share/opensearch-dashboards/config/opensearch_dashboards.yml
RUN echo "  - \"script-src 'unsafe-eval' 'unsafe-inline' 'self'  https://www.gstatic.com/ https://d3js.org/ https://cdn.jsdelivr.net/ https://cdnjs.cloudflare.com/ https://cdn.datatables.net/ 'self'\"" >> /usr/share/opensearch-dashboards/config/opensearch_dashboards.yml
RUN echo "  - \"worker-src blob: *\"" >> /usr/share/opensearch-dashboards/config/opensearch_dashboards.yml
RUN echo "  - \"child-src data: * blob: *\"" >> /usr/share/opensearch-dashboards/config/opensearch_dashboards.yml

# transform_viz
RUN /usr/share/opensearch-dashboards/bin/opensearch-dashboards-plugin install https://github.com/lguillaud/osd_transform_vis/releases/download/$VERSION/transformVis-$VERSION.zip
...

Lionel

Any useful information from the OSD logs ?

Thanks for all the help.

I was just running the base image from docker hub. I have to create a local image with all these settings. Once I do that I’ll respond either way.

Hello,
In order to help you, please find below:

  • Docker file for OS: Dockerfile-os
  • Docker file for OSD: Dockerfile-osd
  • docker-compose file: docker-compose-opensearch.yml

Dockerfile-os

FROM opensearchproject/opensearch:latest

ENV discovery.type=single-node
ENV bootstrap.memory_lock=true
ENV cluster.name=elasticsearch
ENV ES_JAVA_OPTIONS="-Xms1g -Xmx1g"

USER opensearch

# Add log
RUN mkdir -p /usr/share/opensearch/logs
RUN echo "" >> /usr/share/opensearch/config/opensearch.yml

# expose port number
EXPOSE 9200

# Optional conf
RUN echo "http.cors.enabled : true" >>/usr/share/opensearch/config/opensearch.yml
RUN echo "http.cors.allow-origin: \"*\"" >>/usr/share/opensearch/config/opensearch.yml
RUN echo "http.cors.allow-methods : OPTIONS, HEAD, GET, POST, PUT, DELETE" >>/usr/share/opensearch/config/opensearch.yml
RUN echo "http.cors.allow-headers: \"kbn-version, Origin, X-Requested-With, Content-Type, Accept, Engaged-Auth-Token, Authorization\"" >>/usr/share/opensearch/config/opensearch.yml

USER opensearch

Dockerfile-osd

FROM opensearchproject/opensearch-dashboards

EXPOSE 5601

# install required packages to install Kibana & custom plugins
USER root
RUN yum -y update
RUN yum -y install wget

# add opensearch-dashboards logs
RUN mkdir -p /usr/share/opensearch-dashboards/logs
RUN touch /usr/share/opensearch-dashboards/logs/opensearch_dashboards.log
RUN chown -R opensearch-dashboards: /usr/share/opensearch-dashboards/logs
RUN chmod 777 /usr/share/opensearch-dashboards/logs/opensearch_dashboards.log
RUN echo "logging.dest: /usr/share/opensearch-dashboards/logs/opensearch_dashboards.log"  >> /usr/share/opensearch-dashboards/config/opensearch_dashboards.yml

USER opensearch-dashboards
# install opensearch-dashboards plugins
RUN echo "" >> /usr/share/opensearch-dashboards/config/opensearch_dashboards.yml

# Transform viz settings
RUN echo "csp.strict: false" >> /usr/share/opensearch-dashboards/config/opensearch_dashboards.yml
RUN echo "csp.warnLegacyBrowsers: false" >> /usr/share/opensearch-dashboards/config/opensearch_dashboards.yml
RUN echo "csp.rules:" >> /usr/share/opensearch-dashboards/config/opensearch_dashboards.yml
RUN echo "  - \"script-src 'unsafe-eval' 'unsafe-inline' 'self'  https://www.gstatic.com/ https://d3js.org/ https://cdn.jsdelivr.net/ https://cdnjs.cloudflare.com/ https://cdn.datatables.net/ 'self'\"" >> /usr/share/opensearch-dashboards/config/opensearch_dashboards.yml
RUN echo "  - \"worker-src blob: *\"" >> /usr/share/opensearch-dashboards/config/opensearch_dashboards.yml
RUN echo "  - \"child-src data: * blob: *\"" >> /usr/share/opensearch-dashboards/config/opensearch_dashboards.yml

# transform_viz
RUN /usr/share/opensearch-dashboards/bin/opensearch-dashboards-plugin install https://github.com/lguillaud/osd_transform_vis/releases/download/1.2.0/transformVis-1.2.0.zip

# Change network opensearch-dashboards settings
RUN sed -i "s?opensearch.hosts: \[https://localhost:9200\]?opensearch.hosts: \[\"https://opensearch-1.2.4:9200\"\]?" /usr/share/opensearch-dashboards/config/opensearch_dashboards.yml

USER 0
RUN chmod 777 /usr/share/opensearch-dashboards/*

USER opensearch-dashboards

docker-compose-opensearch.yml

version: '3'
services:
  opensearch:
    image: os:latest
    container_name: "opensearch-1.2.4"
    environment:
      - discovery.type=single-node
      - cluster.name=opensearch
      - node.name=opensearch
      - bootstrap.memory_lock=true
      - "ES_JAVA_OPTS=-Xms512m -Xmx512m"
    ulimits:
      memlock:
        soft: -1
        hard: -1
      nofile:
        soft: 65536
        hard: 65536
    ports:
      - "9200:9200"
  opensearch-dashboards:
    image: osd:latest
    container_name: "opensearch-dashboards-1.2.0"
    ports:
      - "5601:5601"

Then run the following commands:

docker build -t os -f Dockerfile-os .
docker build -t osd -f Dockerfile-osd .
docker-compose -f docker-compose-opensearch.yml up

Finally, open localhost:5601 (user and password are admin/Admin).

Hope it helps.
Lionel

1 Like

Wow, thanks for the help Lionel. I was able to get it installed and up and running. I was able to create a map on the dashboard using the samples you provided. I need to figure out the layering next as I want to be able to include points on the graph that can be included or excluded by the user of the dashboard.

Hello,
For adding/removing layers, the leaflet documentation will help you (e.g. Documentation - Leaflet - a JavaScript library for interactive maps). I think you can add buttons (or event click to the map) to handle adding/removing layers.
Lionel

Thanks for the information Lionel. I think this is outside my skillset. I am not sure that what I am trying to do can be done with OpenSearch Dashboards, which is a real problem. I don’t get why OpenSearch is a step backwards from ES. That question is not directed at you, by the way. You have been very helpful.

You’re welcome.
I know this plugin requires Javascript skills but allows great flexibility.
To give a quick answer to the question that was not directed to me :slight_smile:: basically, OpenSearch is a “step backwards from ES” mainly because some of the pieces in ES are proprietary code (under license) and could not be forked as such into OS.
However, depending on the community needs, a map module could come to life …

Lionel, thank you for the answer to the question I did not direct to you :grinning:. I completely understand why it is the way it is. My issue/frustration is that I do not know Javascript. Perhaps everyone else that will run into the problem is Javascript proficient and the need for mapping is not great. In looking at your plugin more closely, I see the before_render and after_render in the map_openstreetmap_custom_geojson example. I guess my question relates to if these get called to update the map based on some change by the user. Let’s say I put a button on the page to add markers based on a query of an index. What method has to get invoked, after_render? Sorry for all the questions. Thanks.

Hello John,

Before_render and after_render hooks are mainly used in the following context:

  • before_render: when you need to manipulate/enrich data that you get from the DSL query part. It ensures that all the data will be ready before rendering the visualization (usually made within the after_render_function). For example, you need to add a field that is computed from the data you get from OS.
  • after_render: when you need to make sure the DOM is built before using it. In other words, wait for the template part (HML/CSS) to be completely interpreted by the browser. This is is the recommended way of doing but you could name the function differently.

The way the plugin works. When the viz is loaded from a dashboard:

  1. A query is made to OpenSearch (if you are using a query in the DSL part)
  2. The template part is built (to render the HTML part)
  3. “At the same time”, the Javascript functions are run (again the after_render function is only triggered when the template is fully loaded).

Each time you change filters/dates, the viz is reloaded and functions are played again.

To answer your questions, what you could do in your case is to add a button and event to handle it in the after_render function.
Add the button in the template part:

<link rel="stylesheet" href="https://cdn.jsdelivr.net/npm/bootstrap@4.5.3/dist/css/bootstrap.min.css" integrity="sha384-TX8t27EcRE3e/ihU7zmQxVncDAy5uIKz4rEkgIXeMed4M0jlfIDPvg6uqKI2xXr2" crossorigin="anonymous">
 <button class='btn btn-primary active' id="myBtn">Change view/map</button>

Add the button management in the javascript part (note that you need to get the button location using the getShadowDomLocation function):

const shadowMyButtonLocation = getShadowDomLocation("#myBtn").vizLocation[0];
$(shadowMyButtonLocation).click(function() {
// insert here what you need to do when you click the button
alert("Hello, I'm here");
});

I know this plugin is not the easiest one as it requires skills in JS development.

Hope it helps a little bit anyway.

Lionel

How to remove a layer ?
In the map_openstreetmap_custom_geojson viz, replace JS part by:

// Leaflet use
// https://nouvelle-techno.fr/articles/pas-a-pas-inserer-une-carte-openstreetmap-sur-votre-site

// Custom geojson
// https://leafletjs.com/examples/geojson/

({
  getShadowDomLocation: function(selector) {
  let vizLocation;
  // output-viz being the top selector used by the Transform plugin
  // Get all the output-viz elements (can have mutliple selector if multiple transfrm vizs in a dashboard)
  // selector parameter value must be unque within the DOM
  const elements = $('.output-vis');
  let shadow;
  for (let elem of elements) {
      shadow = elem.shadowRoot;
      vizLocation = $(shadow).find(selector);
      if (vizLocation.length > 0) {
      // selector found, exiting
        break;
      } else {
        vizLocation = '.notFound';
      }
  } 
  const obj = {
    vizLocation: vizLocation,
    shadowRoot: shadow
    } 
// obj object contains the shadowRoot element and and the location of the selector within the shadowRoot 
  return obj;
},

css: function() {  
  const css_list = [
    "https://use.fontawesome.com/releases/v5.6.3/css/all.css",
    "https://unpkg.com/leaflet@1.7.1/dist/leaflet.css"
  ];  
  for (let css_file of css_list) {
    const links = document.head.querySelectorAll("link");

    // Already been injected
    for(let l in links) if(links[l].href == css_file) return;
      const link = document.createElement('link');
      link.rel = "stylesheet";
      link.href = css_file;
      document.head.appendChild(link);
    } 
  },

  before_render: function() {
    // Set geo points
    let locationsForLayer1 = [
      {
  			"id": "1",
  			"name": "Paris",
  			"lat": "48.8566",
  			"lon": "2.3522",
  			"size": "4",
  			"class": "myDivIconPurple"
  		},
  		{
  			"id": "3",
  			"name": "Toulouse",
  			"lat": "43.6045",
  			"lon": "	1.4440",
  			"size": "2",
  			"class": "myDivIconPurple"
  		}, {
  			"id": "4",
  			"name": "Grenoble",
  			"lat": "45.1715",
  			"lon": "5.7224",
  			"size": "3",
  			"class": "myDivIconPurple"
  		}
    ]
    let locationsForLayer2 = [
      {
  			"id": "2",
  			"name": "Nice",
  			"lat": "43.7034",
  			"lon": "	7.2663",
  			"size": "2",
  			"class": "myDivIcon"
  		},
  		{
  			"id": "5",
  			"name": "Bordeaux",
  			"lat": "44.8400",
  			"lon": "-0.5800",
  			"size": "2",
  			"class": "myDivIcon"
  		}
    ]
    
    response = {
      "locationsForLayer1": locationsForLayer1,
      "locationsForLayer2": locationsForLayer2
    }
  },
  
  after_render: function() {
    const meta = this.meta; 
    let locationsForLayer1 = response.locationsForLayer1;
    let locationsForLayer2 = response.locationsForLayer2;

    $.getScript("https://cdn.jsdelivr.net/npm/leaflet@1.7.1/dist/leaflet.js")
    .done(function(script, textStatus) {
      $.getScript("https://cdn.jsdelivr.net/gh/lguillaud/osd_transform_vis@main/geojson/departements.geojson.js")
      .done(function(script, textStatus) {

        // Get geojson into json object
        let myGeoJson = JSON.parse(departements);

        // Create HTML container for map
        // Rebuild each time to avoid multiple maps layers
        let myDiv = meta.getShadowDomLocation("#myDiv").vizLocation; 
        $(myDiv).empty();
        $(myDiv).append('<div id="map"></div>')
        let mapLocation = meta.getShadowDomLocation("#map").vizLocation[0];
  
        // Create map object
        let map=new L.map(mapLocation);
        
        // Add GeoJSON layer to map
        // Set options
        let myStyle = {
          "color": "#ff7800",
          "weight": 3,
          "opacity": 0.15
        };
        L.geoJSON(myGeoJson, { style: myStyle }).addTo(map);
        
        // Maps location
        L.tileLayer('https://{s}.tile.openstreetmap.org/{z}/{x}/{y}.png', { 
          attribution: 'data © <a href="//osm.org/copyright">OpenStreetMap</a>/ODbL - render <a href="//openstreetmap.fr">OSM France</a>',
          minZoom: 1,
      		maxZoom: 20
      	}).addTo(map); 
  
        // Create markers group
        let markerGroup1 = L.layerGroup();
        let markerGroup2 = L.layerGroup();
        
        // Markers creation layer 1
        let markers = [];  
        for (let location in locationsForLayer1) { 
          // Icon to use
          let myIcon = L.divIcon({
      			html: '<i class="fas fa-circle fa-' + locationsForLayer1[location].size + 'x"></i>',
      			iconSize: [20, 20],
      			className: locationsForLayer1[location].class
      		}); 
  	
  	      // If custom icon
          let marker = L.marker([locationsForLayer1[location].lat, locationsForLayer1[location].lon], { icon: myIcon })
          // Default icon
          //let marker = L.marker([locations[location].lat, locations[location].lon]) 
            .addTo(map)
            .bindPopup(locationsForLayer1[location].name); 
               
          // Add marker to group
          markerGroup1.addLayer(marker);
          // Add marker to list of markers 
  		    markers.push(marker);
        }
    
        // Markers creation layer 2
        markers = [];  
        for (let location in locationsForLayer2) { 
          // Icon to use
          let myIcon = L.divIcon({
      			html: '<i class="fas fa-circle fa-' + locationsForLayer2[location].size + 'x"></i>',
      			iconSize: [20, 20],
      			className: locationsForLayer2[location].class
      		}); 
  	
  	      // If custom icon
          let marker = L.marker([locationsForLayer2[location].lat, locationsForLayer2[location].lon], { icon: myIcon })
          // Default icon
          //let marker = L.marker([locations[location].lat, locations[location].lon]) 
            .addTo(map)
            .bindPopup(locationsForLayer2[location].name); 
               
          // Add marker to group
          markerGroup2.addLayer(marker);
          // Add marker to list of markers 
  		    markers.push(marker);
        }

        // Create marker group for zoom 
        let group = new L.featureGroup(markers);
        // Make all markers visible, using pad to avoid marker being cut 
        map.fitBounds(group.getBounds().pad(0.3));
        map.addLayer(markerGroup1);
        map.addLayer(markerGroup2);
        
        // Manage Button
        const shadowMyButtonLocation = meta.getShadowDomLocation("#myBtn").vizLocation[0];
        $(shadowMyButtonLocation).unbind()
        $(shadowMyButtonLocation).click(function() {
           map.removeLayer(markerGroup2);
        })
      });    
    });    
  }
})

In the template part, replace all by:

<style>
  #map { height: 600px; }  
  
  .myDivIcon {
    text-align: center; /* Horizontally center the text (icon) */
    line-height: 20px; /* Vertically center the text (icon) */
    color: blue;
  }
    
  .myDivIconPurple {
    text-align: center; /* Horizontally center the text (icon) */
    line-height: 20px; /* Vertically center the text (icon) */
    color: purple;
  }
</style>

<link rel="stylesheet" href="https://unpkg.com/leaflet@1.7.1/dist/leaflet.css"
   integrity="sha512-xodZBNTC5n17Xt2atTPuE1HxjVMSvLVW9ocqUKLsCC5CXdbqCmblAshOMAS6/keqq/sMZMZ19scR4PsZChSR7A=="
   crossorigin=""/>

<link rel="stylesheet" href="https://use.fontawesome.com/releases/v5.6.3/css/all.css">

<link rel="stylesheet" href="https://cdn.jsdelivr.net/npm/bootstrap@4.5.3/dist/css/bootstrap.min.css" integrity="sha384-TX8t27EcRE3e/ihU7zmQxVncDAy5uIKz4rEkgIXeMed4M0jlfIDPvg6uqKI2xXr2" crossorigin="anonymous">

<div id="myDiv">
</div>

<button class='btn btn-primary active' id="myBtn">Remove blue points</button>
1 Like

Hi Lionel. Thank you for all the help. I was able to do what I wanted thanks to your plugin and the running help you have provided over the past several days. Your plugin worked to do exactly as I needed.

Hi John,

Glad to hear that.

Again thanks for all the help Lionel. One followup. If I have an OS Dashboard with two visualizations, one controls where the user can set some of the query parameters and the other a transform map, how can I access those query parameter values in the transform map?

What kind of visualization is the one that allows the user to set some of the query parameters ?

If this is also a transform viz, the best would be to have a single transform viz with a part in charge of modifying query parameters and the other part in charge of displaying the map. Being in the same viz, you can add event that re-run what needed.

If this is an out-of-the-box viz (pie, …), when you set new filters it should update the data (query) in the transform viz but only in the case you add the following in the query part (see GitHub - lguillaud/osd_transform_vis: OpenSearch-Dashboards plugin to create custom visualisations):

  • “DASHBOARD_CONTEXT”,
  • “TIME_RANGE[your_time_field_for_the_index_pattern]”