Integrating An Existing Solr Schema

There are two options to generate the Panl configuration file, namely:

  1. Using the Panl Generator
  2. Manually creating the configuration files

Tips: The Panl generator utility is the easiest and quickest method to generate a file.

Using the Panl Generator

IMPORTANT: You cannot  generate the Panl configuration files if your Solr collection starts with the string panl-.  This is a reserved collection prefix for the sole use of Synapticloop Panl.

Running the Command Line Utility

Included within the Panl package is an interactive quick start generator for working with existing managed-schema.xml files for Solr.

It quickly generates a panl.properties file and a <panl_collection_url>.panl.properties file, and links the two.

This command line utility can be invoked with the following commands (ensure that you are in the PANL_INSTALL_DIRECTORY when running the commands).

*NIX command

Command(s)

bin/panl generate ↩
 -schema src/dist/sample/solr/book-store/managed-schema.xml ↩
 -properties src/dist/sample/panl/book-store/
panl.properties


Windows command

Command(s)

bin\panl.bat generate ↩
 -schema src\dist\sample\solr\book-store\managed-schema.xml ↩
 -properties
src\dist\sample\panl\book-store\panl.properties

Running the command for your operating system will output the following to the console, prompting for input for the various properties that are required (with sensible defaults suggested).

Note: Output has had the logging timestamp removed and process designator removed.

01

02

03

04

05

06

07

08

09

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

25

26

27

28

29

30

31

32

33

34

35

36

INFO  panl.Main -             ~ ~ ~ * ~ ~ ~

INFO  panl.Main -

INFO  panl.Main -                            __

INFO  panl.Main -        .-----.---.-.-----.|  |

INFO  panl.Main -        |  _  |  _  |     ||  |

INFO  panl.Main -        |   __|___._|__|__||__|

INFO  panl.Main -        |__|     ... .-..      

INFO  panl.Main -

INFO  panl.Main -             ~ ~ ~ * ~ ~ ~

INFO  panl.Main -

INFO  panl.Main -          Panl version: 1.3.0

INFO  panl.Main -

INFO  panl.Main -     Designed for integration with

INFO  panl.Main -            Solr version: 9

INFO  panl.Main -

INFO  panl.Main -             ~ ~ ~ * ~ ~ ~

INFO  panl.Main -

INFO  panl.Main - Starting Panl generation with properties:

INFO  panl.Main -   -properties src/dist/sample/panl/book-store/panl.properties

INFO  panl.Main -       -schema ↩

src/dist/sample/solr/book-store/managed-schema.xml

INFO  panl.Main -    -overwrite false

INFO  panl.Main -

INFO  panl.Main -             ~ ~ ~ * ~ ~ ~

INFO  panl.Main -

Enter the 1 character property value for 'panl.param.query' ↩

(The search query parameter), default [q]:

Property 'panl.param.query' set to default value of 'q'

Enter the 1 character property value for 'panl.param.page' ↩

(The page number), default [p]:

Property 'panl.param.page' set to default value of 'p'

Enter the 1 character property value for 'panl.param.numrows' ↩

(The number of results to return per page), default [n]:

Property 'panl.param.numrows' set to default value of 'n'

Enter the 1 character property value for 'panl.param.sort' ↩

(The results sorting parameter), default [s]:

Property 'panl.param.sort' set to default value of 's'

Enter the 1 character property value for 'panl.param.operand' ↩
(The default query operand (q.op)), default [o]:

Property 'panl.param.operand' set to default value of 'o'

Enter the 1 character property value for 'panl.param.passthrough' ↩

(The URL path passthrough), default [z]:

Property 'panl.param.passthrough' set to default value of 'z'

This will output the file to the src\dist\sample\panl\book-store\ directory and will generate two files, the panl.properties file and the <panl_collection_url>.panl.properties. From here you may edit the files and set the configuration for your specific use case.

Lines 17-21:

The Panl generator will output the passed in command line options so that you can confirm that the input and output files are correct

Lines 25-36:

Will prompt for input from the console.  This will check to ensure that

  1. There is only one character for the Panl params, and
  2. That the character has not been used before

If the enter key is pressed with an empty response, then the default value will be used.

If there are any errors, the generator will print out the error and re-prompt to enter the parameter.

The Panl generator will output its working to the command line and highlight any warnings.  A reduced version of the output is below:

01

02

03

04

05

06

07

08

09

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

INFO  bean.PanlCollection - PanlCollection: book-store

INFO  bean.PanlCollection - Have 17 panlFields, LPSE length is set to 1

INFO  bean.PanlCollection - Assigned field 'id' to panl code 'i'

INFO  bean.PanlCollection - Assigned field 'author' to panl code 'a'

INFO  bean.PanlCollection - Assigned field 'title' to panl code 't'

INFO  bean.PanlCollection - Assigned field 'description' to panl code 'd'

INFO  bean.PanlCollection - Assigned field 'book_image' to panl code 'b'

INFO  bean.PanlCollection - Assigned field 'buy_url' to panl code 'B'

INFO  bean.PanlCollection - Assigned field 'genre' to panl code 'g'

INFO  bean.PanlCollection - Assigned field 'num_pages' to panl code 'N'

INFO  bean.PanlCollection - Assigned field 'first_published_year' to panl code 'f'

INFO  bean.PanlCollection - Assigned field 'language' to panl code 'l'

INFO  bean.PanlCollection - Assigned field 'is_paperback' to panl code 'I'

INFO  bean.PanlCollection - Assigned field 'series' to panl code 'S'

INFO  bean.PanlCollection - Assigned field 'price' to panl code 'P'

INFO  bean.PanlCollection - Assigned field 'a_to_z_index' to panl code 'A'

INFO  bean.PanlCollection - Assigned field 'decade_published' to panl code 'D'

WARN  bean.PanlCollection - No nice panl code for field 'book_length',  ↩

'b' and 'B' already taken

INFO  bean.PanlCollection - Assigned field 'text' to panl code 'T'

INFO  bean.PanlCollection - Assigned field 'book_length' to RANDOM panl code 'u'

INFO  generator.PanlGenerator - Writing out file panl.properties

INFO  generator.PanlGenerator - Done writing out file panl.properties

INFO  generator.PanlGenerator - Writing out file book-store.panl.properties

INFO  generator.PanlGenerator - Done writing out file book-store.panl.properties

Line 18 and 20:

This is a warning from the generator that it could not automatically assign a LPSE code to the Solr field book_length.  What the generator does is look at the first character of the Solr field, and attempt to assign it to that character as a LPSE code, first as a lowercase letter, then as an uppercase letter.  

If neither are available, it will assign it a random LPSE code from the available codes - in this case book_length was assigned the LPSE code of 'u'.

Editing the Generated Files

The panl.properties file that is generated is well commented and you will definitely need to edit the solrj.client and solr.search.server.url properties.

Also, if running in production mode, it is recommended that you set all of the following properties to false:

  • panl.results.testing.urls
  • panl.status.404.verbose
  • panl.status.500.verbose

Moving on to the <panl_collection_url>.panl.properties file, use the properties quick reference section to edit the file to assign prefixes, suffixes, set the type of the facet to a field, or OR, BOOLEAN, RANGE etc.

Manually Creating the Configuration Files

Manually creating the Panl configuration files is not the recommended path as it:

  • Can be time consuming,
  • Is prone to errors,
  • Cross-checking of values and LPSE codes is required

However, for small data sets, it is a good way to fully understand how the configuration works.

You can use the existing files as a starting point, or look at the templates in the github repository.

The panl.properties File

The panl.properties file is quite straight-forward with only a few configuration options that are required to be set.

The property requires a little thought and understanding is the Panl collections properties.

All Panl collections properties should be placed at the end of the file and have the format:

panl.collection.<solr_collection_name>=<properties_file_location>

Where:

  • <solr_collection_name> is the collection to query on the Solr server
  • <properties_file_location> is the relative location FROM this file (i.e. this panl.properties file).  

The format of the <properties_file_location> is:

  • <panl_collection_uri>.panl.properties

Where:

  • <panl_collection_uri> is the base URI path that the Panl server will respond to, i.e. it will be bound to the Panl URL of http://localhost:port/<panl_collection_uri>

NOTE: You may have multiple <panl_collection_uri> values for each <solr_collection_name> with different Panl collections and fieldset configurations (CaFUPs).

The example mechanical pencils panl.properties file:

https://github.com/synapticloop/panl/blob/main/src/dist/sample/panl/mechanical-pencils/panl.properties 

The template upon which it was based:

https://github.com/synapticloop/panl/blob/main/src/main/resources/panl.properties.template

The <panl_collection_url>.panl.properties File

These files configure the Panl collection and the CaFUPs for the URLs to be served by the Panl server.

This is a far more complex file which configures the fields and facets to be returned with the results, along with any configuration options for individual fields or facets.

The configuration options depend on what the Solr field type is and, without using the generator, can become difficult.

The best way to manually generate the properties is to use the example file below and change the configuration as necessary.  Each of the below files contain expansive commenting which will help you through the configuration.

The example mechanical-pencils.panl.properties file:

https://github.com/synapticloop/panl/blob/main/src/dist/sample/panl/mechanical-pencils/mechanical-pencils.panl.properties 

The template upon which it was based:

https://github.com/synapticloop/panl/blob/main/src/main/resources/panl_collection_url.panl.properties.template

IMPORTANT: Ensure that you are using the correct version of the file for both your Panl and Solr installation.  Whilst the properties files should be backwards compatible, there may be additional configuration items which are enabled in later versions and will have no effect if configured for previous versions.

For the latest Panl version, with the Solr 8 connection properties can be found

https://github.com/synapticloop/panl/tree/solr-panl-8/src/main/resources

For the latest Panl version, with the Solr 7 connection properties can be found

https://github.com/synapticloop/panl/tree/solr-panl-7/src/main/resources

~ ~ ~ * ~ ~ ~