Managing Elasticsearch indices without repeating the same work over and over again.
Remap your indices without any notable downtime for your users. Elasticorn will automatically select the correct index and copy your data.
Apply the same configuration principles across multiple applications. Your developers will immediately feel at home.
Keep your configuration where you would look for it, point Elasticorn to your config directory and let the magic happen.
Whenever you need to adjust the mapping of your elasticsearch index, all data has to be deleted.
This makes total sense because when inserting a new document into elasticsearch this document gets analyzed based on your current mapping. So changing the mapping needs this process to be restarted again.
So if you have a lot of data in your index, changing the mapping could result in serious downtime for your users - simply because you need to re-import and/or re-index all your current data.
Elasticsearch suggests to work with two indices, one being the LIVE index and the other being the standby index.
The names of these indices do not matter, because you would let your application "talk" to Elasticsearch using an alias which points to your LIVE index.
If you need to change the mapping of your data you would do so on the standby index, re-import all your data and then switch the alias to your former standby index.
Whenever you write an app that uses elasticsearch you are faced with the same challenges.
If you are doing it "right" you are also adding a couple of more things to your app:
The risk of letting out any of the points above is pretty high - leading to a sub-par environment. Running error prone setups cost time and more importantly: nerves.
by wrapping all these tasks into a meaningful configuration and a couple of command line tasks that you won't want to miss once you've used them.
So what is convenience when working in teams?
Our definition is that your team should focus on creating
great applications with elasticsearch. You should not fiddle
around different code-bases copying and pasting things
around.
Ideally all your projects should use the same
structure of configuration and behave in a predictable
manner.
In short: Manage your Elasticsearch indices with Elasticorn and you will not want to go back.
What if you have a running app from 2 years ago you now want to manage with Elasticorn?
No worries, Elasticorn can:
You see, that's what we call "convenience".
Download both the phar and the pubkey file to the same folder. Call ./elasticorn.phar and you are good to go.
local: composer require t3g/elasticorn
global: composer global require t3g/elasticorn
Elasticorn assumes default connection parameters for establishing a connection to elasticsearch. If you are using a non-default setup you can configure those connection settings in a .env file. For details see below.
Elasticorn is compatible with both elasticsearch 2.x and 5.x. For 2.x take elasticorn 1, for 5.x use elasticorn 5.
For elasticorn to work, your configuration needs to be structured in the following way and be defined as yaml.
- MAIN configuration directory
- IndexName directory
- IndexConfiguration.yaml
- DocumentTypes directory
- documenttype.yaml (for example: tweets.yaml)
project-folder
│ README.md
└───Elasticorn
└── t3_forger
├── DocumentTypes
│ ├── issue.yaml
│ ├── review.yaml
│ └── user.yaml
└── IndexConfiguration.yaml
In our case the Elasticorn
holds all information
about our indices. Multiple indices can be managed by
creating new folders.
The IndexConfiguration.yaml
file specifies
configuration parameters for the index (for example shards or
replicas.)
The folder called DocumentTypes
holds our type
mapping with a file per document type.
The syntax is pretty straightforward yaml syntax which will then be parsed as an array.
The filename will determine the name of your document type in Elasticsearch.
We'll take a look at user.yaml
here:
id:
type: integer
username:
type: string
index: not_analyzed
store: true
fullname:
type: string
index: not_analyzed
store: true
email:
type: integer
index: not_analyzed
store: true
avatar:
type: string
For an example on how the configuration should look like, see the Tests/Fixtures/Configuration folder in this project. For a list of available configuration options see the elastica documentation.
You can use a .env file, a command line parameter or the interactive console to specify your configuration directory.
You can specify your configuration directory as well as specific connection params in a .env file which should be placed in the folder where elasticorn gets executed. The following variables may be configured:
configurationPath=
ELASTICA_HOST=
ELASTICA_PORT=
ELASTICA_PATH=
ELASTICA_URL=
ELASTICA_TRANSPORT=
ELASTICA_PERSISTENT=
ELASTICA_TIMEOUT=
ELASTICA_USERNAME=
ELASTICA_PASSWORD=
composer based usage command:
./elasticorn.php -h
phar usage command:
./elasticorn.phar -h
Emanuel-Leutze-Str. 11,
40547 Düsseldorf, Germany.
Mon - Fri : 9am to 6pm