• logo

    Explore Elasticorn

    Your time-saving, zero-downtime elasticsearch index manager written in PHP7.

About Elasticorn

Managing Elasticsearch indices without repeating the same work over and over again.

Zero Downtime

Remap your indices without any notable downtime for your users. Elasticorn will automatically select the correct index and copy your data.

Save time

Apply the same configuration principles across multiple applications. Your developers will immediately feel at home.

Convenient usage

Keep your configuration where you would look for it, point Elasticorn to your config directory and let the magic happen.


Zero Downtime

The problem

Whenever you need to adjust the mapping of your elasticsearch index, all data has to be deleted.

This makes total sense because when inserting a new document into elasticsearch this document gets analyzed based on your current mapping. So changing the mapping needs this process to be restarted again.

So if you have a lot of data in your index, changing the mapping could result in serious downtime for your users - simply because you need to re-import and/or re-index all your current data.


The Solution

Elasticsearch suggests to work with two indices, one being the LIVE index and the other being the standby index.

The names of these indices do not matter, because you would let your application "talk" to Elasticsearch using an alias which points to your LIVE index.

If you need to change the mapping of your data you would do so on the standby index, re-import all your data and then switch the alias to your former standby index.

100%

happy users

200%

happy devs

300%

happy admins

Save time

Tired of doing the same things over and over again in your apps?

Whenever you write an app that uses elasticsearch you are faced with the same challenges.

  • Setting up an index for your local development environment
  • Setting up (the same) index for your staging system
  • Setting the same index up (a 3rd time) for your live environment
  • Configuring your mappings

If you are doing it "right" you are also adding a couple of more things to your app:

  • Setting aliases
  • Making all things configurable
  • Setting up a live and standby index
  • Write a service that will apply mappings, copy over the data, clean up the old mess.

The risk of letting out any of the points above is pretty high - leading to a sub-par environment. Running error prone setups cost time and more importantly: nerves.

Elasticorn saves you a lot of time

by wrapping all these tasks into a meaningful configuration and a couple of command line tasks that you won't want to miss once you've used them.

Convenience for your team

So what is convenience when working in teams?

Our definition is that your team should focus on creating great applications with elasticsearch. You should not fiddle around different code-bases copying and pasting things around.
Ideally all your projects should use the same structure of configuration and behave in a predictable manner.

This is why we created Elasticorn.

  • All your configuration is in one place.
  • Everyone on your team knows where to look for it.
  • Close-to-none configuration (in fact, we allow you to configure the path to your configuration so your devs have to type less :))
  • Strong conventions!
  • Reliable tooling

In short: Manage your Elasticsearch indices with Elasticorn and you will not want to go back.

But there's more

What if you have a running app from 2 years ago you now want to manage with Elasticorn?

No worries, Elasticorn can:

  • Take the current configuration of your app
  • Create config files from it
  • Set up your new live and standby indices
  • Apply the configured mapping
  • Copy your data over to the new live index
  • Rename your "old" index (for backup purposes)
  • Set the alias for your app

You see, that's what we call "convenience".

Documentation

Installation

Install .phar

Download both the phar and the pubkey file to the same folder. Call ./elasticorn.phar and you are good to go.

Install via composer

local: composer require t3g/elasticorn
global: composer global require t3g/elasticorn

Elasticsearch Client Configuration

Elasticorn assumes default connection parameters for establishing a connection to elasticsearch. If you are using a non-default setup you can configure those connection settings in a .env file. For details see below.

Elasticsearch Compatibility

Elasticorn is compatible with both elasticsearch 2.x and 5.x. For 2.x take elasticorn 1, for 5.x use elasticorn 5.

Index and Mapping Configuration

For elasticorn to work, your configuration needs to be structured in the following way and be defined as yaml.

- MAIN configuration directory
  - IndexName directory
    - IndexConfiguration.yaml
    - DocumentTypes directory
      - documenttype.yaml (for example: tweets.yaml)

Example

project-folder
	│ README.md
	└───Elasticorn
		└── t3_forger
			├── DocumentTypes
			│   ├── issue.yaml
			│   ├── review.yaml
			│   └── user.yaml
			└── IndexConfiguration.yaml

In our case the Elasticorn holds all information about our indices. Multiple indices can be managed by creating new folders.

The IndexConfiguration.yaml file specifies configuration parameters for the index (for example shards or replicas.)

The folder called DocumentTypes holds our type mapping with a file per document type.

The syntax is pretty straightforward yaml syntax which will then be parsed as an array.

The filename will determine the name of your document type in Elasticsearch.

We'll take a look at user.yaml here:

id:
  type: integer
username:
  type: string
  index: not_analyzed
  store: true
fullname:
  type: string
  index: not_analyzed
  store: true
email:
  type: integer
  index: not_analyzed
  store: true
avatar:
  type: string

For an example on how the configuration should look like, see the Tests/Fixtures/Configuration folder in this project. For a list of available configuration options see the elastica documentation.

You can use a .env file, a command line parameter or the interactive console to specify your configuration directory.

.env configuration

You can specify your configuration directory as well as specific connection params in a .env file which should be placed in the folder where elasticorn gets executed. The following variables may be configured:

configurationPath=
ELASTICA_HOST=
ELASTICA_PORT=
ELASTICA_PATH=
ELASTICA_URL=
ELASTICA_TRANSPORT=
ELASTICA_PERSISTENT=
ELASTICA_TIMEOUT=
ELASTICA_USERNAME=
ELASTICA_PASSWORD=

Usage

composer based usage command:

./elasticorn.php -h

phar usage command:

./elasticorn.phar -h

Download

Download Source

Get Source

Download as PHAR

Get PHAR Get Public Key


Address

Emanuel-Leutze-Str. 11,
40547 Düsseldorf, Germany.

office Hours

Mon - Fri : 9am to 6pm

Legal Disclosure

Information in accordance with section 5 TMG

TYPO3 GmbH
Emanuel-Leutze-Str. 11
40547 Düsseldorf

Represented by

Mathias Schreiber

Contact

Telephone: 0175 93 86 731
E-Mail: info@typo3.com
Internetaddress: typo3.com

Register entry

Entry in Handelsregister
Register Number: 77950
Register Court: Amtsgericht Düsseldorf

Capital

Initial or original capital: 25000.00 €

Disclaimer

Accountability for content
The contents of our pages have been created with the utmost care. However, we cannot guarantee the contents' accuracy, completeness or topicality. According to statutory provisions, we are furthermore responsible for our own content on these web pages. In this context, please note that we are accordingly not obliged to monitor merely the transmitted or saved information of third parties, or investigate circumstances pointing to illegal activity. Our obligations to remove or block the use of information under generally applicable laws remain unaffected by this as per §§ 8 to 10 of the Telemedia Act (TMG).

Accountability for links
Responsibility for the content of external links (to web pages of third parties) lies solely with the operators of the linked pages. No violations were evident to us at the time of linking. Should any legal infringement become known to us, we will remove the respective link immediately.

Copyright
Our web pages and their contents are subject to German copyright law. Unless expressly permitted by law (§ 44a et seq. of the copyright law), every form of utilizing, reproducing or processing works subject to copyright protection on our web pages requires the prior consent of the respective owner of the rights. Individual reproductions of a work are allowed only for private use, so must not serve either directly or indirectly for earnings. Unauthorized utilization of copyrighted works is punishable (§ 106 of the copyright law).

This website uses Google Analytics, a web analytics service provided by Google, Inc. (“Google”). Google Analytics uses “cookies”, which are text files placed on your computer, to help the website analyze how users use the site. The information generated by the cookie about your use of the website (including your IP address) will be transmitted to and stored by Google on servers in the United States. In case of activation of the IP anonymization, Google will truncate/anonymize the last octet of the IP address for Member States of the European Union as well as for other parties to the Agreement on the European Economic Area. Only in exceptional cases, the full IP address is sent to and shortened by Google servers in the USA. On behalf of the website provider Google will use this information for the purpose of evaluating your use of the website, compiling reports on website activity for website operators and providing other services relating to website activity and internet usage to the website provider. Google will not associate your IP address with any other data held by Google. You may refuse the use of cookies by selecting the appropriate settings on your browser. However, please note that if you do this, you may not be able to use the full functionality of this website. Furthermore you can prevent Google’s collection and use of data (cookies and IP address) by downloading and installing the browser plug-in available under https://tools.google.com/dlpage/gaoptout?hl=en-GB. You can refuse the use of Google Analytics by clicking on the following link. An opt-out cookie will be set on the computer, which prevents the future collection of your data when visiting this website:

Disable Google Analytics

Further information concerning the terms and conditions of use and data privacy can be found at http://www.google.com/analytics/terms/gb.html or at https://www.google.de/intl/en_uk/policies/. Please note that on this website, Google Analytics code is supplemented by “anonymizeIp” to ensure an anonymized collection of IP addresses (so called IP-masking).