JDONREFv4 Plugin : Différence entre versions

De JDONREF Wiki
(→‎Includes)
 
(37 révisions intermédiaires par le même utilisateur non affichées)
Ligne 1 : Ligne 1 :
 
The JDONREF plugin provides a way to index adresses compatible with french format and search them in a single line format, with 100% accurate results and great order.
  
The JDONREF plugin provides a way to index adresses compatible with french format and search them in a single line format, with 100% accurate results and great order.
   
−
It works over the provided PayloadCheckerSpanQuery that target to combines whatever work rules who want (not only adresses) to search data in a single line format, with 100% accurate results and great order.
 +
It works over the provided PayloadCheckerSpanQuery that target to combines whatever work rules you want (not only adresses) to search data in a single line format, with 100% accurate results and great order.
 
Scorers are adaptable for human or computer uses.
  
Scorers are adaptable for human or computer uses.
  +
  +
Hercule had better watch out !
   
 
===== Plugin installation =====
  
===== Plugin installation =====
   
−
Simply run at the root of you ''ElasticSearch v1.3+'' installation :
 +
Simply run at the root of your ''ElasticSearch v1.7'' installation :
   
−
bin/plugin --install elasticsearch-jdonrefv3es-0.2beta --url http://sourceforge.net/p/jdonref/code/HEAD/tree/Dev/JDONREF_ES_PLUGIN_POC/dist/elasticsearch-jdonrefv3es-0.2beta.zip?format=raw
 +
bin/plugin --install elasticsearch-jdonrefv4-0.5 --url http://sourceforge.net/p/jdonref/code/HEAD/tree/Dev/JDONREFv4/dist/elasticsearch-jdonrefv4-0.5.zip?format=raw
   
 
This will download the last beta version plugin from sourceforge.
  
This will download the last beta version plugin from sourceforge.
Ligne 16 : Ligne 18 :
   
 
You can follow [[JDONREFv3ES_plugin_GettingStarted|the getting started guide]] for jdonref plugin.
  
You can follow [[JDONREFv3ES_plugin_GettingStarted|the getting started guide]] for jdonref plugin.
−
The same will arise soon for PayloadCheckerSpanQuery.
  
   
 
There is an embedded site in order to test the query like this :
  
There is an embedded site in order to test the query like this :
   
−
http://url_elasticsearch:port/_plugin/jdonrefv4-0.2beta/JDONREFv3Test.html
 +
http://url_elasticsearch:port/_plugin/jdonrefv4-0.5/JDONREFv4Test.html
−  
−
It only supports index named 'jdonref' for now.
  
−
Of course there is no such limitation for jdonrefv4 nor PayloadCheckerSpanQuery.
  
   
 
===== Releases =====
  
===== Releases =====
   
 
{| border="1"
  
{| border="1"
−
| '''JDONREFv3ES plugin'''
+
| '''JDONREF plugin'''
 
| '''elasticsearch'''
 
| '''elasticsearch'''
 
| '''release date'''
  
| '''release date'''
Ligne 37 : Ligne 38 :
 
| 0.2beta
  
| 0.2beta
 
| 1.1.X
  
| 1.1.X
−
| 2014-08-25 (see 0.2)
 +
| 2014-08-25
 
| adress based score with integration tests
  
| adress based score with integration tests
 
|-
  
|-
 
| 0.2
  
| 0.2
 
| '''1.3.X'''
  
| '''1.3.X'''
−
| 2014-10-30 (still improving performances)
  
  +
| 2014-11-15
−
| production version : adress based score with production test.
 +
| production version : adress based score with production tests. human validation only. miss bulk scorer and explain.
 
|-
  
|-
 
| 0.3
  
| 0.3
 
| 1.3.X
  
| 1.3.X
−
| 2015-03-31
 +
| 2015-12-15
−
| score tuning
 +
| reduce high frequency term frequencies (improve performances)
  +
|-
  +
| 0.4beta
  +
| 1.3.X
  +
| 2015-05-15
  +
| minimum_should_match
  +
|-
  +
| 0.4
  +
| 1.3.X
  +
| 2015-12-31
  +
| bulk scorer & minimum_should_match. Performance Problems need to be solved again ! Waiting for ES support.
  +
|-
  +
| 0.5beta
  +
| 1.7.X
  +
| 2016-02-26
  +
| Performances. Need a big tuning by now !
  +
|-
  +
| 0.5
  +
| 1.7.X
  +
| 2016-06-31
  +
| final wildcard, fuzzyness, general performances
 
|}
  
|}
   
Ligne 60 : Ligne 61 :
 
| [[JDONREFv3ES_Query|jdonrefv4]]
  
| [[JDONREFv3ES_Query|jdonrefv4]]
 
| query/scorer
  
| query/scorer
−
| 0.2beta (weights and term can't be changed). Read [[PayloadCheckerSpanQuery|PayloadCheckerSpanQuery]].
 +
| since 0.2beta (weights and term can't be changed for now).
 
|-
  
|-
−
| [[PayloadCheckerSpanQuery|PayloadCheckerSpanQuery]]
  
  +
| [[PayloadCheckerSpanFilter|PayloadCheckerSpanFilter]]
−
| query/scorer
  
  +
| filter
−
| 0.2beta (grammar filter based on payloads used by jdonrefv4).
 +
| since 0.4beta (grammar filter based on payloads, not used by jdonrefv4 any more).
 
|-
  
|-
−
| [[JDONREFv4_Mappings#Le_mapping_poizon|POI et Zones]]
  
  +
| [[UnsplitFilter|UnsplitFilter]]
−
| mapping
  
  +
| token filter
−
| 0.2beta
  
  +
| since 0.3beta (reduce tokens frequencies, not used by jdonrefv4 any more).
 
|-
  
|-
−
| [[JDONREFv4_Mappings#Le_mapping_adresse|adresse]]
  
  +
| [[JDONREFv4_Mappings#Le_mapping_poizon|POI et Zones]], [[JDONREFv4_Mappings#Le_mapping_adresse|adresse]], [[JDONREFv4_Mappings#Le_mapping_troncon|troncon]], [[JDONREFv4_Mappings#Le_mapping_voie|voie]], [[JDONREFv4_Mappings#Le_mapping_commune|commune]], [[JDONREFv4_Mappings#Le_mapping_departement|departement]], [[JDONREFv4_Mappings#Le_mapping_pays|pays]]
 
| mapping
  
| mapping
−
| 0.2beta
 +
| since 0.2beta
−
|-
  
−
| [[JDONREFv4_Mappings#Le_mapping_troncon|troncon]]
  
−
| mapping
  
−
| 0.2beta
  
−
|-
  
−
| [[JDONREFv4_Mappings#Le_mapping_voie|voie]]
  
−
| mapping
  
−
| 0.2beta
  
−
|-
  
−
| [[JDONREFv4_Mappings#Le_mapping_commune|commune]]
  
−
| mapping
  
−
| 0.2beta
  
−
|-
  
−
| [[JDONREFv4_Mappings#Le_mapping_departement|departement]]
  
−
| mapping
  
−
| 0.2beta
  
−
|-
  
−
| [[JDONREFv4_Mappings#Le_mapping_pays|pays]]
  
−
| mapping
  
−
| 0.2beta
  
 
|-
  
|-
 
| [[JDONREFv4_synonymes|jdonrefv3es_synonym.fr.txt]]
  
| [[JDONREFv4_synonymes|jdonrefv3es_synonym.fr.txt]]
 
| French Synonyms'file
  
| French Synonyms'file
−
| 0.1beta
 +
| since 0.1beta
 
|-
  
|-
−
| jdonrefv4-settings.json
 +
| [[jdonrefv4-settings.json]]
 
| settings
  
| settings
−
| 0.2beta jdonrefv4_index, jdonrefv4_search, jdonrefv4_codes_index
 +
| since 0.2beta jdonrefv4_index, jdonrefv4_search, jdonrefv4_codes_index
 
|-
  
|-
 
| JDONREFv4TermSimilarity
  
| JDONREFv4TermSimilarity
 
| similarity
  
| similarity
−
| 0.2beta
 +
| since 0.2beta
 
|-
  
|-
 
| [[EdgeNGramWithPayloadsFilter]]
  
| [[EdgeNGramWithPayloadsFilter]]
 
| token filter
  
| token filter
−
| 0.2beta (edgengram with payloads support)
 +
| since 0.2beta (edgengram with payloads support, not used by jdonrefv4 any more)
 
|-
  
|-
 
| [[TokenCountPayloadFilter]]
  
| [[TokenCountPayloadFilter]]
 
| token filter
  
| token filter
−
| 0.2beta (insert in payload the count of token with given payload - for use by PayloadVersusTypeSpanQuery).
 +
| since 0.2beta (insert the number of token with given payload in each such payload - used by PayloadCheckerSpanQuery#All).
 
|-
  
|-
−
| [[GroupedPayloadSpanQuery]]
  
  +
| [[SynonymWithPayloadsFilter]]
−
| query
  
  +
| token filter
−
| 0.2beta (already deprecated see payloadcheckerspanQuery, remove results where payloads are not grouped together).
  
  +
| since 0.2beta (synonym with payloads support, not used by jdonrefv4 any more).
 
|-
  
|-
−
| [[PayloadVersusTypeSpanQuery]]
  
  +
| JDONREFv3Test.html
−
| query
  
−
| 0.2beta (already deprecated see payloadcheckerspanQuery, remove results where all payloads term matches for given types).
  
−
|-
  
−
| [[JDONREFv3ES_Type|jdonrefv3es_adresse]]
  
−
| type
  
−
|
 
−
|-
  
−
| [[JDONREFv3ES_AdressTokenFilter|jdonrefv3es_adresse]]
  
−
| tokenFilter
  
−
|
 
−
|-
  
−
| [[JDONREFv3ES_filter|combosynonym]]
  
−
| TokenFilter
  
−
|
 
−
|-
  
−
| test_0.html test_1.html
  
 
| site
  
| site
−
| 0.1beta (simple test)
  
  +
| since 0.2beta (site for jdonrefv3es and index jdonref) Set 10000 for maxSizePerType with dedicated index of 5 shards for each type !
 
|-
  
|-
−
| JDONREFv3Test.html
 +
| JDONREFv4Test.html
 
| site
  
| site
−
| 0.2beta (site for jdonrefv3es and index jdonref)
  
  +
| since 0.4. Set 10000 for maxSizePerType with dedicated index of 5 shards for each type !
 
|}
  
|}
   

Version actuelle en date du 26 février 2016 à 23:05

The JDONREF plugin provides a way to index adresses compatible with french format and search them in a single line format, with 100% accurate results and great order.

It works over the provided PayloadCheckerSpanQuery that target to combines whatever work rules you want (not only adresses) to search data in a single line format, with 100% accurate results and great order. Scorers are adaptable for human or computer uses.

Hercule had better watch out !

Plugin installation

Simply run at the root of your ElasticSearch v1.7 installation :

bin/plugin --install elasticsearch-jdonrefv4-0.5 --url http://sourceforge.net/p/jdonref/code/HEAD/tree/Dev/JDONREFv4/dist/elasticsearch-jdonrefv4-0.5.zip?format=raw

This will download the last beta version plugin from sourceforge.

Don't forget restart elasticsearch.


You can follow the getting started guide for jdonref plugin.

There is an embedded site in order to test the query like this :

http://url_elasticsearch:port/_plugin/jdonrefv4-0.5/JDONREFv4Test.html

Releases
JDONREF plugin elasticsearch release date detail
0.1beta 1.0.0.RC1 2014-04-30 POC
0.2beta 1.1.X 2014-08-25 adress based score with integration tests
0.2 1.3.X 2014-11-15 production version : adress based score with production tests. human validation only. miss bulk scorer and explain.
0.3 1.3.X 2015-12-15 reduce high frequency term frequencies (improve performances)
0.4beta 1.3.X 2015-05-15 minimum_should_match
0.4 1.3.X 2015-12-31 bulk scorer & minimum_should_match. Performance Problems need to be solved again ! Waiting for ES support.
0.5beta 1.7.X 2016-02-26 Performances. Need a big tuning by now !
0.5 1.7.X 2016-06-31 final wildcard, fuzzyness, general performances
Includes
name type disponibilité
jdonrefv4 query/scorer since 0.2beta (weights and term can't be changed for now).
PayloadCheckerSpanFilter filter since 0.4beta (grammar filter based on payloads, not used by jdonrefv4 any more).
UnsplitFilter token filter since 0.3beta (reduce tokens frequencies, not used by jdonrefv4 any more).
POI et Zones, adresse, troncon, voie, commune, departement, pays mapping since 0.2beta
jdonrefv3es_synonym.fr.txt French Synonyms'file since 0.1beta
jdonrefv4-settings.json settings since 0.2beta jdonrefv4_index, jdonrefv4_search, jdonrefv4_codes_index
JDONREFv4TermSimilarity similarity since 0.2beta
EdgeNGramWithPayloadsFilter token filter since 0.2beta (edgengram with payloads support, not used by jdonrefv4 any more)
TokenCountPayloadFilter token filter since 0.2beta (insert the number of token with given payload in each such payload - used by PayloadCheckerSpanQuery#All).
SynonymWithPayloadsFilter token filter since 0.2beta (synonym with payloads support, not used by jdonrefv4 any more).
JDONREFv3Test.html site since 0.2beta (site for jdonrefv3es and index jdonref) Set 10000 for maxSizePerType with dedicated index of 5 shards for each type !
JDONREFv4Test.html site since 0.4. Set 10000 for maxSizePerType with dedicated index of 5 shards for each type !

See also todo list.

Récupérée de « https://adullact.net/plugins/mediawiki/wiki/jdonref-v2/index.php?title=JDONREFv4_Plugin&oldid=484 »