-
Explore Elasticorn
Your time-saving, zero-downtime elasticsearch index manager written in PHP7.
About Elasticorn
Managing Elasticsearch indices without repeating the same work over and over again.
Zero Downtime
Remap your indices without any notable downtime for your users. Elasticorn will automatically select the correct index and copy your data.
Save time
Apply the same configuration principles across multiple applications. Your developers will immediately feel at home.
Convenient usage
Keep your configuration where you would look for it, point Elasticorn to your config directory and let the magic happen.
Zero Downtime
The problem
Whenever you need to adjust the mapping of your elasticsearch index, all data has to be deleted.
This makes total sense because when inserting a new document into elasticsearch this document gets analyzed based on your current mapping. So changing the mapping needs this process to be restarted again.
So if you have a lot of data in your index, changing the mapping could result in serious downtime for your users - simply because you need to re-import and/or re-index all your current data.
The Solution
Elasticsearch suggests to work with two indices, one being the LIVE index and the other being the standby index.
The names of these indices do not matter, because you would let your application "talk" to Elasticsearch using an alias which points to your LIVE index.
If you need to change the mapping of your data you would do so on the standby index, re-import all your data and then switch the alias to your former standby index.
Save time
Tired of doing the same things over and over again in your apps?
Whenever you write an app that uses elasticsearch you are faced with the same challenges.
- Setting up an index for your local development environment
- Setting up (the same) index for your staging system
- Setting the same index up (a 3rd time) for your live environment
- Configuring your mappings
If you are doing it "right" you are also adding a couple of more things to your app:
- Setting aliases
- Making all things configurable
- Setting up a live and standby index
- Write a service that will apply mappings, copy over the data, clean up the old mess.
The risk of letting out any of the points above is pretty high - leading to a sub-par environment. Running error prone setups cost time and more importantly: nerves.
Elasticorn saves you a lot of time
by wrapping all these tasks into a meaningful configuration and a couple of command line tasks that you won't want to miss once you've used them.
Convenience for your team
So what is convenience when working in teams?
Our definition is that your team should focus on creating
great applications with elasticsearch. You should not fiddle
around different code-bases copying and pasting things
around.
Ideally all your projects should use the same
structure of configuration and behave in a predictable
manner.
This is why we created Elasticorn.
- All your configuration is in one place.
- Everyone on your team knows where to look for it.
- Close-to-none configuration (in fact, we allow you to configure the path to your configuration so your devs have to type less :))
- Strong conventions!
- Reliable tooling
In short: Manage your Elasticsearch indices with Elasticorn and you will not want to go back.
But there's more
What if you have a running app from 2 years ago you now want to manage with Elasticorn?
No worries, Elasticorn can:
- Take the current configuration of your app
- Create config files from it
- Set up your new live and standby indices
- Apply the configured mapping
- Copy your data over to the new live index
- Rename your "old" index (for backup purposes)
- Set the alias for your app
You see, that's what we call "convenience".
Documentation
Installation
Install .phar
Download both the phar and the pubkey file to the same folder. Call ./elasticorn.phar and you are good to go.
Install via composer
local: composer require t3g/elasticorn
global: composer global require t3g/elasticorn
Elasticsearch Client Configuration
Elasticorn assumes default connection parameters for establishing a connection to elasticsearch. If you are using a non-default setup you can configure those connection settings in a .env file. For details see below.
Elasticsearch Compatibility
Elasticorn is compatible with both elasticsearch 2.x and 5.x. For 2.x take elasticorn 1, for 5.x use elasticorn 5.
Index and Mapping Configuration
For elasticorn to work, your configuration needs to be structured in the following way and be defined as yaml.
- MAIN configuration directory
- IndexName directory
- IndexConfiguration.yaml
- DocumentTypes directory
- documenttype.yaml (for example: tweets.yaml)
Example
project-folder
│ README.md
└───Elasticorn
└── t3_forger
├── DocumentTypes
│ ├── issue.yaml
│ ├── review.yaml
│ └── user.yaml
└── IndexConfiguration.yaml
In our case the Elasticorn holds all information
about our indices. Multiple indices can be managed by
creating new folders.
The IndexConfiguration.yaml file specifies
configuration parameters for the index (for example shards or
replicas.)
The folder called DocumentTypes holds our type
mapping with a file per document type.
The syntax is pretty straightforward yaml syntax which will then be parsed as an array.
The filename will determine the name of your document type in Elasticsearch.
We'll take a look at user.yaml here:
id:
type: integer
username:
type: string
index: not_analyzed
store: true
fullname:
type: string
index: not_analyzed
store: true
email:
type: integer
index: not_analyzed
store: true
avatar:
type: string
For an example on how the configuration should look like, see the Tests/Fixtures/Configuration folder in this project. For a list of available configuration options see the elastica documentation.
You can use a .env file, a command line parameter or the interactive console to specify your configuration directory.
.env configuration
You can specify your configuration directory as well as specific connection params in a .env file which should be placed in the folder where elasticorn gets executed. The following variables may be configured:
configurationPath=
ELASTICA_HOST=
ELASTICA_PORT=
ELASTICA_PATH=
ELASTICA_URL=
ELASTICA_TRANSPORT=
ELASTICA_PERSISTENT=
ELASTICA_TIMEOUT=
ELASTICA_USERNAME=
ELASTICA_PASSWORD=
Usage
composer based usage command:
./elasticorn.php -h
phar usage command:
./elasticorn.phar -h
Download
Download Source
Download as PHAR
Address
Emanuel-Leutze-Str. 11,
40547 Düsseldorf, Germany.
office Hours
Mon - Fri : 9am to 6pm