Getting Started With Panl

A rather pleasing companion to the Apache® Solr® Faceted Search Engine

_____________

Author: Synapticloop - Version 1.1.1


This book will get you up and running with a fully featured, SEO friendly, keyword searchable, faceted search engine with an in-built, example search page to test it all out.


(Note: The most up-to-date version of this book can always be found here: https://github.com/synapticloop/panl/tree/main/src/book) 

WHY?

Because...


/Caran+d'Ache/true/Black/bDW/


looks so much nicer than...


q=*:*&facet.mincount=1&rows=10&facet.field=lead_size_indicator&facet.fie
ld=grip_material&facet.field=
colours&facet.field=nib_shape&facet.field=d
iameter&facet.field=cap_shape&facet.field=brand&facet.field=mechanism_ty
pe&facet.field=length&facet.field=hardness_indicator&facet.field=grip_ty
pe&facet.field=cap_material&facet.field=lead_grade_indicator&facet.field
=tubing_material&facet.field=in_built_sharpener&facet.field=
disassemble&
facet.field=category&facet.field=body_shape&facet.field=clip_material&fa
cet.field=mechanism_material&facet.field=lead_length&facet.field=body_ma
terial&facet.field=in_built_eraser&facet.field=grip_shape&facet.field=re
lative_weight&facet.field=
name&facet.field=nib_material&facet.field=weig
ht&facet.field=
variants&facet=true&fq=brand:"Caran+d'Ache"&fq=disassembl
e:"true"
&fq=colours:"Black"&q.op=AND



TL;DR

If you are looking to get the quickest start on understanding how a Panl server is configured and up and running, then...

  1. Start with the Quick Start - The 5 Steps section to get the Solr and Panl servers up and running.
  2. Look at the Panl Results Viewer
    http://localhost:8181/panl-results-viewer/mechanical-pencils/default/ 
  3. Work through the A Walkthrough Example - The Book Store section to gain a deeper understanding of the configuration options.
  4. Use your existing dataset with  Integrating An Existing Solr Schema
  5. Need help? See the Panl Configuration section


QUICK TIP

If you need to restart the example Solr server installation - try these commands:

Windows:

Command(s)

cd SOLR_INSTALL_DIRECTORY

bin\solr start -cloud -p 8983 -s "example\cloud\node1\solr"

bin\solr start -cloud -p 7574 -s "example\cloud\node2\solr" -z localhost:9983

*NIX:

Command(s)

cd SOLR_INSTALL_DIRECTORY

bin/solr start -cloud -p 8983 -s "example/cloud/node1/solr"

bin/solr start -cloud -p 7574 -s "example/cloud/node2/solr" -z localhost:9983

Tip: Documentation for this book can also be read online, see

https://synapticloop.github.io/panl/, or to download the latest copy of this book, see https://github.com/synapticloop/panl/releases/.

Copyright Notice

The data used in this book is based on 2mm mechanical pencil data hand curated by the author.  The data and associated Solr search results utilised in this book references manufacturers of writing instruments, specifically 2mm mechanical pencils.  The manufacturer's name, the name of the pencil (and/or model number) is a trademark and/or  is copyright of the manufacturer.

Additional datasets may have information that is copyright to the respective owners.

Whilst the project is released under the very liberal MIT licence, the sample mechanical data remains under the copyright of Synapticloop, and may not be used for commercial purposes.

~ ~ ~ * ~ ~ ~


TABLE OF CONTENTS

README.1ST        13

Welcome To Synapticloop Panl        13

Additional Panl Niceties        13

About This Book        15

Nomenclature Used Throughout This Book        15

Book Format Conventions        18

About Panl Server        20

How Many Facets Does Panl Support?        21

In-Built Web Apps (Viewer / Explainer / Single Search Page)        22

About Panl Generator        30

About Apache Solr        30

Why Synapticloop Panl?        32

What is a LPSE (pronounced 'lapse') code?        32

The three types of LPSE codes        33

URL Path Nomenclature        34

Is Synapticloop Panl For Me?        34

What You Will Need        36

Download The Resources        36

The Panl Directory Structure        36

Panl Server Versions        39

Solr-Panl-9-x.x.x        39

Solr-Panl-8-x.x.x & Solr-Panl-7-x.x.x        39

Solr-Panl-6-x.x.x and earlier version        40

Quick Start - The 5 Steps        41

A Note On Running The Commands        42

Windows Commands        43

*NIX Commands        44

Next Steps        46

Getting Started        47

Downloading the Resources        47

The Solr server        48

The Panl server        48

Creating and Starting a Solr Cloud Instance        48

Creating a collection        51

Indexing the Data        52

Starting The Panl Server        53

When Something Goes Wrong        54

Restarting the Solr Panl tutorial process        55

Stopping Solr        55

Deleting a collection        56

Re-Starting Solr        56

Worst-case scenario        57

Next Steps        57

Additional Data        58

All Data Panl Server        58

Simple Date        58

Creating and Indexing the Data        58

panl.properties File Additions        59

Mechanical Pencils OR Facet        59

Creating and Indexing the Data        60

panl.properties File Additions        60

Book Store        60

A Walkthrough Example - The Book Store        62

0. High Level Requirements        63

1. Understanding the Dataset        64

2. Configure the Solr Index        66

3. Configure the Panl Server        71

The Default panl.properties file        72

The Default Search Page Configuration        73

4. Determine the Web Pages to Render        90

Author and Author Series        90

Author Listing        90

The Iterative Implementation Process        91

Working With Any Dataset        92

Supported Solr Data Types        92

Unsupported/Partially Supported Solr Field Types        93

Facet and Field Types        94

Fields        94

Regular Facets        95

BOOLEAN Facets        96

RANGE Facets        96

DATE Range Facets        97

OR Facets        99

Other Facet Options        100

A Brief Introduction To Solr        102

Querying Data        102

The Solr Managed Schema        103

Determining the Appropriate FieldType and Attributes        107

The Solr Configuration File        111

Query Request Handler        111

Highlighting        113

Panl Configuration        114

The panl.properties file        114

The <panl_collection_url>.panl.properties file        115

The panl.properties Configuration File        117

Configuring The SolrJ Connector        118

Setting The Solr Server URL(s)        119

Enabling The Panl Results Testing URLs        120

The Panl Results Viewer        121

The Panl Results Explainer        122

The Panl Single Page Search        122

Setting HTTP Status Message Verbosity        123

Binding Solr collections to Panl URL paths        124

The <panl_collection_url>.panl.properties Configuration File        125

Parameter and Operand Definitions        125

panl.param.query        125

panl.param.sort        126

panl.param.page        127

panl.param.numrows        128

panl.param.query.operand        128

panl.param.passthrough        129

panl.param.passthrough.canonical        130

panl.form.query.respondto        132

panl.include.single.facets        132

panl.include.same.number.facets        132

solr.default.query.operand        134

solr.facet.limit        134

solr.facet.min.count        134

solr.numrows.default        134

solr.highlight        135

Field Definitions        135

Properties Available for Facets        135

Facet Definitions        136

Properties Available for Facets        137

Regular Facet        140

Example        141

Decoding the URL        142

BOOLEAN Facet        142

Additional Properties        143

Example        144

Decoding the URL Without a Prefix or Suffix        144

Defining a Prefix and Suffix        145

Decoding the URL With a Prefix or Suffix        145

OR Facet        146

Additional Properties        146

The Difference Between Multi-valued Facets and OR Facets        146

Example        150

Decoding the URL        152

RANGE Facet        153

Additional Properties        154

Visualising the Properties        156

Decoding the URL - Range facet with an infix        158

Decoding the URL - Range facet without an infix        158

DATE Range Facet        159

Additional Properties        162

Example        162

Decoding the URL        163

Some Notes        164

About Value Replacements        164

On Field Validations        166

About Hierarchical Facets        166

About Sorting Facets        167

Putting It All Together (Technically)        169

A Brief Architecture        169

Generating SEO Friendly URLs        170

Data Dependencies        170

Passthrough URLs        170

Handling Errors        171

Panl LPSE URL Paths Explained        172

Parameter Query        173

Parameter Query Operand        174

Parameter Number of Rows        174

Parameter Page Number        176

Parameter Pass Through        177

Parameter Sort Order        178

Facets        179

Explaining A More Complex Example        181

Search Integration And The Panl Response Object        183

URL Bindings        183

Error Responses        183

Success Responses        184

The "active" JSON Object        185

Overall Integration and Implementation        187

The "active.facet" JSON Array (Regular facet object)        188

Integration and Implementation        189

The "active.facet" JSON Array (BOOLEAN facet object)        189

Integration and Implementation        190

The "active.facet" JSON Array (DATE Range facet object)        191

Integration and Implementation        192

The "active.facet" JSON Array (OR facet object)        193

Integration and Implementation        194

The "active.facet" JSON Array (RANGE facet object)        194

Integration and Implementation        195

The "active.numrows" JSON Object        196

Integration and Implementation        196

The "active.page" JSON Object        196

Integration and Implementation        197

The "active.query" JSON Object        197

Integration and Implementation        198

The "active.query" JSON Object        198

Integration and Implementation        199

The "active.sort" JSON Array        199

Integration and Implementation        199

The "available" JSON Object        200

The "available.date_range_facets" JSON Array        201

Integration and Implementation        202

The "available.facets" JSON Array        203

Integration and Implementation        204

The "available.range_facets" JSON Object        204

Implementation and Integration        205

The "sorting" JSON Object        212

Implementation and Integration        213

Single Search Page Integration And The Panl Response Object        218

URL Bindings        218

Error Responses        219

Success Responses        219

The "lpse_lookup" JSON Object        220

The "lpse_order" JSON Array        221

Generating the LPSE URL path        221

Integrating An Existing Solr Schema        222

Using the Panl Generator        222

Running the Command Line Utility        222

Editing the Generated Files        226

Manually Creating the Configuration Files        226

Properties Quick Reference        228

Property References Format Explained        228

panl.bool.<lpse_code>.false        229

panl.bool.<lpse_code>.true        230

panl.collection.<solr_collection_name>        231

panl.date.<lpse_code>.days        232

panl.date.<lpse_code>.hours        233

panl.date.<lpse_code>.months        234

panl.date.<lpse_code>.next        235

panl.date.<lpse_code>.previous        236

panl.date.<lpse_code>.years        238

panl.decimal.point        239

panl.facet.<lpse_code>        239

panl.facetsort.<lpse_code>        240

panl.field.<lpse_code>        241

panl.form.query.respondto        242

panl.include.same.number.facets        242

panl.include.single.facets        243

panl.lpse.ignore        243

panl.lpse.length        244

panl.lpse.order        244

panl.multivalue.<lpse_code>        245

panl.name.<lpse_code>        246

panl.or.facet.<lpse_code>        247

panl.param.numrows        247

panl.param.numrows.prefix        248

panl.param.numrows.suffix        248

panl.param.page        249

panl.param.page.prefix        249

panl.param.page.suffix        250

panl.param.passthrough        250

panl.param.passthrough.canonical        251

panl.param.query        251

panl.param.query.operand        252

panl.param.sort        252

panl.prefix.<lpse_code>        253

panl.range.facet.<lpse_code>        253

panl.range.infix.<lpse_code>        254

panl.range.max.<lpse_code>        255

panl.range.max.value.<lpse_code>        255

panl.range.max.wildcard.<lpse_code>        256

panl.range.min.<lpse_code>        257

panl.range.min.value.<lpse_code>        257

panl.range.min.wildcard.<lpse_code>        258

panl.range.prefix.<lpse_code>        259

panl.range.suffix.<lpse_code>        260

panl.range.suppress.<lpse_code>        260

panl.results.fields.<field_set>        261

panl.results.fields.default        261

panl.results.fields.empty        262

panl.results.testing.urls        262

panl.results.sort.fields        263

panl.status.404.verbose        263

panl.status.500.verbose        264

panl.suffix.<lpse_code>        264

panl.type.<lpse_code>        265

panl.when.<lpse_code>        265

solr.default.query.operand        266

solr.facet.limit        266

solr.facet.min.count        267

solr.highlight        267

solr.numrows.default        268

solr.search.server.url        268

solrj.client        268

Panl Cookbook        270

amazon.com.au        271

argos.co.uk        272

realestate.com.au        273

BookFinder.com        274

Rymans.co.uk        275

Design Considerations        277

Collections And FieldSet URL Paths (CaFUPs)        278

In-Built Panl Results Testing URLs        279

Panl Server Startup        280

Error Messaging        281

404 - Not Found        281

500 - Internal Server Error        282

Testing vs. Production        284

Quick Middleware        284

Afterword        285

On The Tag-Line        285

On Documentation        285

Additional Functionality in the Pipeline        287

Appendices        290

Definitions        290

Command Line Options        295

Panl Server        295

Panl Generator        295

Usage Text        296

Sample .properties Files        299

The Sample panl.properties Files        299

The Sample <panl_collection_url>.panl.properties Files        300

Solr Version 8 & 7 Integration Notes        301

SolrJ        301

Solr Configuration files        301

JSON Response Object        301

Setting up a Solr 7 or 8 server        302

Additional Solr Version 7 Integration Notes        307

Solr Versions 6 and below        308