JDONREFv4 Plugin : Différence entre versions

De JDONREF Wiki
(→‎Includes)
(→‎Includes)
 
(78 révisions intermédiaires par le même utilisateur non affichées)
Ligne 1 : Ligne 1 :
−
L'intérêt du plugin JDONREFv3ES pour ElasticSearch est multiple. Il fournit :
  
  +
The JDONREF plugin provides a way to index adresses compatible with french format and search them in a single line format, with 100% accurate results and great order.
−
* les [[JDONREFv3ES_Mappings|mappings]] utilisés par JDONREF ;
  
  +
−
* un [[JDONREFv3ES_synonymes|fichier de synonymes]] adapté à la norme Française ;
  
  +
It works over the provided PayloadCheckerSpanQuery that target to combines whatever work rules you want (not only adresses) to search data in a single line format, with 100% accurate results and great order.
−
* un [[JDONREFv3ES_Type|type adresse]] associé à un [[JDONREFv3ES_AdressTokenFilter|TokenFilter adresse]] permettant d'indexer les adresses pour une recherche plus efficace (gestion des numéros, etc ...) ;
  
  +
Scorers are adaptable for human or computer uses.
−
* une [[JDONREFv3ES_Query|requête]] qui permet un calcul de note adapté à l'homme comme à la machine ;
  
  +
  +
Hercule had better watch out !
  +
  +
===== Plugin installation =====
  +
  +
Simply run at the root of your ''ElasticSearch v1.7'' installation :
  +
  +
bin/plugin --install elasticsearch-jdonrefv4-0.5 --url http://sourceforge.net/p/jdonref/code/HEAD/tree/Dev/JDONREFv4/dist/elasticsearch-jdonrefv4-0.5.zip?format=raw
  +
  +
This will download the last beta version plugin from sourceforge.
  +
  +
Don't forget restart elasticsearch.
  +
  +
  +
You can follow [[JDONREFv3ES_plugin_GettingStarted|the getting started guide]] for jdonref plugin.
  +
  +
There is an embedded site in order to test the query like this :
  +
  +
http://url_elasticsearch:port/_plugin/jdonrefv4-0.5/JDONREFv4Test.html
   
 
===== Releases =====
  
===== Releases =====
   
 
{| border="1"
  
{| border="1"
−
| '''JDONREFv3ES plugin'''
+
| '''JDONREF plugin'''
 
| '''elasticsearch'''
 
| '''elasticsearch'''
 
| '''release date'''
  
| '''release date'''
  +
| '''detail'''
 
|-
  
|-
 
| 0.1beta
  
| 0.1beta
 
| 1.0.0.RC1
 
| 1.0.0.RC1
−
| 2014-04-30
+
| 2014-04-30
  +
| POC
 
|-
  
|-
−
| 0.2 (utilisable en production)
  
  +
| 0.2beta
 
| 1.1.X
  
| 1.1.X
−
| 2014-06-27
+
| 2014-08-25
  +
| adress based score with integration tests
  +
|-
  +
| 0.2
  +
| '''1.3.X'''
  +
| 2014-11-15
  +
| production version : adress based score with production tests. human validation only. miss bulk scorer and explain.
  +
|-
  +
| 0.3
  +
| 1.3.X
  +
| 2015-12-15
  +
| reduce high frequency term frequencies (improve performances)
  +
|-
  +
| 0.4beta
  +
| 1.3.X
  +
| 2015-05-15
  +
| minimum_should_match
  +
|-
  +
| 0.4
  +
| 1.3.X
  +
| 2015-12-31
  +
| bulk scorer & minimum_should_match. Performance Problems need to be solved again ! Waiting for ES support.
  +
|-
  +
| 0.5beta
  +
| 1.7.X
  +
| 2016-02-26
  +
| Performances. Need a big tuning by now !
  +
|-
  +
| 0.5
  +
| 1.7.X
  +
| 2016-06-31
  +
| final wildcard, fuzzyness, general performances
 
|}
  
|}
   
Ligne 28 : Ligne 47 :
 
| '''disponibilité'''
  
| '''disponibilité'''
 
|-
  
|-
−
| [[JDONREFv3ES_Query|jdonrefv3es]]
 +
| [[JDONREFv3ES_Query|jdonrefv4]]
 
| query/scorer
  
| query/scorer
−
| 0.1beta<br>0.2 (poids et éléments non personnalisables).
  
  +
| since 0.2beta (weights and term can't be changed for now).
 
|-
  
|-
−
| [[JDONREFv3ES_Mappings#Le_mapping_adresse|adresse]]
  
  +
| [[PayloadCheckerSpanFilter|PayloadCheckerSpanFilter]]
−
| mapping
 +
| filter
−
| 0.1beta<br>0.2
 +
| since 0.4beta (grammar filter based on payloads, not used by jdonrefv4 any more).
 
|-
  
|-
−
| [[JDONREFv3ES_Mappings#Le_mapping_troncon|troncon]]
  
  +
| [[UnsplitFilter|UnsplitFilter]]
−
| mapping
  
  +
| token filter
−
| 0.1beta<br>0.2
  
  +
| since 0.3beta (reduce tokens frequencies, not used by jdonrefv4 any more).
 
|-
  
|-
−
| [[JDONREFv3ES_Mappings#Le_mapping_voie|voie]]
  
  +
| [[JDONREFv4_Mappings#Le_mapping_poizon|POI et Zones]], [[JDONREFv4_Mappings#Le_mapping_adresse|adresse]], [[JDONREFv4_Mappings#Le_mapping_troncon|troncon]], [[JDONREFv4_Mappings#Le_mapping_voie|voie]], [[JDONREFv4_Mappings#Le_mapping_commune|commune]], [[JDONREFv4_Mappings#Le_mapping_departement|departement]], [[JDONREFv4_Mappings#Le_mapping_pays|pays]]
 
| mapping
  
| mapping
−
| 0.1beta<br>0.2
  
  +
| since 0.2beta
 
|-
  
|-
−
| [[JDONREFv3ES_Mappings#Le_mapping_commune|commune]]
  
  +
| [[JDONREFv4_synonymes|jdonrefv3es_synonym.fr.txt]]
−
| mapping
  
  +
| French Synonyms'file
−
| 0.1beta<br>0.2
 +
| since 0.1beta
 
|-
  
|-
−
| [[JDONREFv3ES_Mappings#Le_mapping_departement|departement]]
  
  +
| [[jdonrefv4-settings.json]]
−
| mapping
  
−
| 0.1beta<br>0.2
  
−
|-
  
−
| [[JDONREFv3ES_Mappings#Le_mapping_pays|pays]]
  
−
| mapping
  
−
| 0.1beta<br>0.2
  
−
|-
  
−
| [[JDONREFv3ES_synonymes|jdonrefv3es_synonym.fr.txt]]
  
−
| fichier de synonymes français
  
−
| 0.1beta
  
−
|-
  
−
| jdonrefv3es-settings.json
  
 
| settings
  
| settings
−
| 0.1beta (jdonrefv3es_index, jdonrefv3es_search)<br>0.2
  
  +
| since 0.2beta jdonrefv4_index, jdonrefv4_search, jdonrefv4_codes_index
 
|-
  
|-
−
| [[JDONREFv3ES_Type|jdonrefv3es_adresse]]
  
  +
| JDONREFv4TermSimilarity
−
| type
  
  +
| similarity
−
|
 
  +
| since 0.2beta
 
|-
  
|-
−
| [[JDONREFv3ES_AdressTokenFilter|jdonrefv3es_adresse]]
  
  +
| [[EdgeNGramWithPayloadsFilter]]
−
| tokenFilter
  
  +
| token filter
−
|
 
  +
| since 0.2beta (edgengram with payloads support, not used by jdonrefv4 any more)
 
|-
  
|-
−
| [[JDONREFv3ES_filter|combosynonym]]
  
  +
| [[TokenCountPayloadFilter]]
−
| TokenFilter
  
  +
| token filter
−
|
 
  +
| since 0.2beta (insert the number of token with given payload in each such payload - used by PayloadCheckerSpanQuery#All).
 
|-
  
|-
−
| test_0.html test_1.html
  
  +
| [[SynonymWithPayloadsFilter]]
−
| site
  
  +
| token filter
−
| 0.1beta (test basique d'un index d'adresse respectant le mapping)
  
  +
| since 0.2beta (synonym with payloads support, not used by jdonrefv4 any more).
 
|-
  
|-
 
| JDONREFv3Test.html
  
| JDONREFv3Test.html
 
| site
  
| site
−
|
 
  +
| since 0.2beta (site for jdonrefv3es and index jdonref) Set 10000 for maxSizePerType with dedicated index of 5 shards for each type !
  +
|-
  +
| JDONREFv4Test.html
  +
| site
  +
| since 0.4. Set 10000 for maxSizePerType with dedicated index of 5 shards for each type !
 
|}
  
|}
   
−
===== Installation =====
  
  +
See also [[JDONREFv3ES_TODO|todo list]].
−  
−
Pour installer le plugin, vous pouvez simplement exécuter la commande suivante :
  
−  
−
bin/plugin --install elasticsearch-jdonrefv3es-0.1beta --url http://sourceforge.net/p/jdonref/code/HEAD/tree/Dev/JDONREF_ES_PLUGIN_POC/dist/elasticsearch-jdonrefv3es-0.1beta.zip?format=raw
  
−  
−
Le plugin n'offre pour le moment qu'une requête et un site de test.
  
−  
−
La requête est utilisable tel que décrit dans [[jdonrefv3es_plugin_configuration|ici]], mais est pour le moment pratiquement aussi utile que l'indexation proposée par le fichier jdonrefv3es-settings.json.
  
−
Une page de test sera fournie pour cette requête ultérieurement.
  
−  
−
Le site est accessible par http://url_elasticsearch:port/_plugin/jdonrefv3es-0.1beta/test_0.html.
  
−
Il fonctionne avec un index de nom "jdonref" qui respecte le mapping proposé par le plugin .
  
−  
−
A cet effet, pour créer un tel index, il s'agit d'utiliser la description fournie par le fichier jdonrefv3es-settings.json et de créer les types correspondant à chaque mapping fourni. Le plugin s'occupera de ces opérations dans les versions à venir.
  
−
Enfin, le fichier jdonrefv3es_synonym.fr.txt doit être déployé dans /etc/elasticsearch ou dans la chaîne du path avant la création de l'index.
  
−  
−
===== Utilisation =====
  
−  
−
Voir notamment le [[JDONREFv3ES_plugin_GettingStarted|guide de première utilisation]].
  

Version actuelle en date du 26 février 2016 à 22:05

The JDONREF plugin provides a way to index adresses compatible with french format and search them in a single line format, with 100% accurate results and great order.

It works over the provided PayloadCheckerSpanQuery that target to combines whatever work rules you want (not only adresses) to search data in a single line format, with 100% accurate results and great order. Scorers are adaptable for human or computer uses.

Hercule had better watch out !

Plugin installation

Simply run at the root of your ElasticSearch v1.7 installation :

bin/plugin --install elasticsearch-jdonrefv4-0.5 --url http://sourceforge.net/p/jdonref/code/HEAD/tree/Dev/JDONREFv4/dist/elasticsearch-jdonrefv4-0.5.zip?format=raw

This will download the last beta version plugin from sourceforge.

Don't forget restart elasticsearch.


You can follow the getting started guide for jdonref plugin.

There is an embedded site in order to test the query like this :

http://url_elasticsearch:port/_plugin/jdonrefv4-0.5/JDONREFv4Test.html

Releases
JDONREF plugin elasticsearch release date detail
0.1beta 1.0.0.RC1 2014-04-30 POC
0.2beta 1.1.X 2014-08-25 adress based score with integration tests
0.2 1.3.X 2014-11-15 production version : adress based score with production tests. human validation only. miss bulk scorer and explain.
0.3 1.3.X 2015-12-15 reduce high frequency term frequencies (improve performances)
0.4beta 1.3.X 2015-05-15 minimum_should_match
0.4 1.3.X 2015-12-31 bulk scorer & minimum_should_match. Performance Problems need to be solved again ! Waiting for ES support.
0.5beta 1.7.X 2016-02-26 Performances. Need a big tuning by now !
0.5 1.7.X 2016-06-31 final wildcard, fuzzyness, general performances
Includes
name type disponibilité
jdonrefv4 query/scorer since 0.2beta (weights and term can't be changed for now).
PayloadCheckerSpanFilter filter since 0.4beta (grammar filter based on payloads, not used by jdonrefv4 any more).
UnsplitFilter token filter since 0.3beta (reduce tokens frequencies, not used by jdonrefv4 any more).
POI et Zones, adresse, troncon, voie, commune, departement, pays mapping since 0.2beta
jdonrefv3es_synonym.fr.txt French Synonyms'file since 0.1beta
jdonrefv4-settings.json settings since 0.2beta jdonrefv4_index, jdonrefv4_search, jdonrefv4_codes_index
JDONREFv4TermSimilarity similarity since 0.2beta
EdgeNGramWithPayloadsFilter token filter since 0.2beta (edgengram with payloads support, not used by jdonrefv4 any more)
TokenCountPayloadFilter token filter since 0.2beta (insert the number of token with given payload in each such payload - used by PayloadCheckerSpanQuery#All).
SynonymWithPayloadsFilter token filter since 0.2beta (synonym with payloads support, not used by jdonrefv4 any more).
JDONREFv3Test.html site since 0.2beta (site for jdonrefv3es and index jdonref) Set 10000 for maxSizePerType with dedicated index of 5 shards for each type !
JDONREFv4Test.html site since 0.4. Set 10000 for maxSizePerType with dedicated index of 5 shards for each type !

See also todo list.

Récupérée de « https://adullact.net/plugins/mediawiki/wiki/jdonref-v2/index.php?title=JDONREFv4_Plugin&oldid=484 »