Can I run multiple Elasticsearch instances in one server?

John Doe ·

1436 Views

 

Yes and I did that recently with my laptop for experimentation with things that I would like to do with ES Cluster. This setup simplifies my environment, instead of using docker or using multiple machines/VMs/EC2 instances.

You can download and use the .tar.gz file recommended above, and then you can do one of the following options

  • option 1 - untar the file into two directories: es-01es-02 then update the config file in each node

  • option 2 - untar the file into one directory: es-instance, copy the config directory to another location with something like path/to/es-01/configpath/to/es-02/config then create a script to launch elasticsearch from the same binary location (which is es-instance) but use different config location (one for es-01/config, one for es-02/config)

You need to update the following parameters in config/elasticsearch.yml file to separate one node from the other.

  • cluster.name
  • node.name
  • path.data
  • path.logs
  • http.port

 

Can I run multiple Elasticsearch instances in one server? -If yes, How can I install 2 Elasticsearch instances in a single Linux server. I am aware if we use "yum install" will be installed without any issue. Thanks in Advance
I have several machines each with 128 GB of ram, each host is running a single instance of Elasticsearch. I would like to run another data node on each host and allocate around 30 GB to the jvm he...

 

You need to prepare two elasticsearch.yml config files to configure settings accordingly and specify these files when startup up the two nodes.

/elasticsearch/config/elasticsearch.yml
/elasticsearch/config2/elasticsearch.yml

/elasticsearch/config2/elasticsearch.yml

cluster.name: trinitarian
node.name: node-2
path.data: /path/to/data
path.logs: path/to/logs
network.host: 127.0.0.1
network.bind_host: 127.0.0.1
network.publish_host: 127.0.0.1
http.port: 9500

 

Linux

Assuming your rpm or deb created an init.d script, to start a second node on the same machine do as follows:

cd /etc/init.d
cp --preserve elasticsearch elasticsearch2

Edit elasticsearch2 script:

  1. change # elasticsearch to # elasticsearch2
  2. add node="2" after line prog="elasticsearch"
  3. change pidfile=/var/run/elasticsearch/${prog}.pid to pidfile=/var/run/elasticsearch/${prog}${node}.pid
  4. change lockfile=/var/lock/subsys/$prog to lockfile=/var/lock/subsys/$prog$node
  5. change echo -n $"Starting $prog: " to echo -n $"Starting $prog: (node $node)"
  6. change echo -n $"Stopping $prog: " to echo -n $"Stopping $prog: (node $node)"

Save the file. Execute

chkconfig --add elasticsearch2
service elasticsearch2 start

I also had to add -Des.config=<path-to-second-config-files> to get it working

 

I have a server running Ubuntu 14.04 with 220 GB of ram on which I'd like to run elasticsearch. According to the documentation, one node should not have more than 32 GB of RAM, so I guess I have to...

 

Windows

/elasticsearch/bin/elasticsearch-env2.bat

if not defined ES_PATH_CONF (
  set ES_PATH_CONF=!ES_HOME!\config2
)

/elasticsearch/bin/elasticsearch2.bat

CALL "%~dp0elasticsearch-env2.bat" || exit /b 1

 

Check this:

http://localhost:9200/

{
  "name" : "BEAST",
  "cluster_name" : "elasticsearch",
  "cluster_uuid" : "EVBl4ttGSWKP8MJJMivMCg",
  "version" : {
    "number" : "7.16.2",
    "build_flavor" : "default",
    "build_type" : "zip",
    "build_hash" : "2b937c44140b6559905130a8650c64dbd0879cfb",
    "build_date" : "2021-12-18T19:42:46.604893745Z",
    "build_snapshot" : false,
    "lucene_version" : "8.10.1",
    "minimum_wire_compatibility_version" : "6.8.0",
    "minimum_index_compatibility_version" : "6.0.0-beta1"
  },
  "tagline" : "You Know, for Search"
}

http://localhost:9500/

{
  "name" : "node-2",
  "cluster_name" : "trinitarian",
  "cluster_uuid" : "QRUfSRYISH-20jU7x9yxog",
  "version" : {
    "number" : "7.16.2",
    "build_flavor" : "default",
    "build_type" : "zip",
    "build_hash" : "2b937c44140b6559905130a8650c64dbd0879cfb",
    "build_date" : "2021-12-18T19:42:46.604893745Z",
    "build_snapshot" : false,
    "lucene_version" : "8.10.1",
    "minimum_wire_compatibility_version" : "6.8.0",
    "minimum_index_compatibility_version" : "6.0.0-beta1"
  },
  "tagline" : "You Know, for Search"
}

 

Rebuild index:

PS E:\pythonProjects\trinitarian\test> python manage.py search_index --rebuild --model xxx
Are you sure you want to delete the 'board' indexes? [y/N]: y
Deleting index 'board'
Creating index 'board'
Indexing 26 'Board' objects
PS E:\pythonProjects\trinitarian\test>

 

Test with this code (test.py):

import elasticsearch

elastic = elasticsearch.Elasticsearch(hosts=["localhost:9200"])
indices = elastic.indices.get_alias().keys()

print(indices)

elastic2 = elasticsearch.Elasticsearch(hosts=["localhost:9500"])
indices2 = elastic2.indices.get_alias().keys()

print(indices2)

 

Result:

E:\pythonProjects\trinitarian\venv\Scripts\python.exe E:/pythonProjects/trinitarian/test/app/test.py
dict_keys(['a', 'b', 'c', 'board'])
dict_keys(['board'])

Process finished with exit code 0

 

elasticsearch