Getting Started With Panl

A rather pleasing companion to the Apache® Solr® Faceted Search Engine

_____________

Author: Synapticloop - Version 1.2.0


This book will rapidly get you up and running with a fully featured, SEO friendly, keyword searchable, faceted search engine with an in-built, example search page to test it all out.


(Note: The most up-to-date version of this book can always be found here: https://github.com/synapticloop/panl/tree/main/src/book) 

WHY?

Because...


/Caran+d'Ache/true/Black/bDW/


looks so much nicer than...


q=*:*&facet.mincount=1&rows=10&facet.field=lead_size_indicator&facet.fie
ld=grip_material&facet.field=
colours&facet.field=nib_shape&facet.field=d
iameter&facet.field=cap_shape&facet.field=brand&facet.field=mechanism_ty
pe&facet.field=length&facet.field=hardness_indicator&facet.field=grip_ty
pe&facet.field=cap_material&facet.field=lead_grade_indicator&facet.field
=tubing_material&facet.field=in_built_sharpener&facet.field=
disassemble&
facet.field=category&facet.field=body_shape&facet.field=clip_material&fa
cet.field=mechanism_material&facet.field=lead_length&facet.field=body_ma
terial&facet.field=in_built_eraser&facet.field=grip_shape&facet.field=re
lative_weight&facet.field=
name&facet.field=nib_material&facet.field=weig
ht&facet.field=
variants&facet=true&fq=brand:"Caran+d'Ache"&fq=disassembl
e:"true"
&fq=colours:"Black"&q.op=AND



TL;DR

If you are looking to get the quickest start on understanding how a Panl server is configured and up and running, then...

  1. Start with the Quick Start - The 5 Steps section to get the Solr and Panl servers up and running.
  2. Look at the Panl Results Viewer
    http://localhost:8181/panl-results-viewer/mechanical-pencils/default/ 
  3. Work through the A Walkthrough Example - The Book Store section to gain a deeper understanding of the configuration options.
  4. Use your existing dataset with  Integrating An Existing Solr Schema
  5. Need help understanding the options? See the Panl Configuration section


QUICK TIP

If you need to restart the example Solr server installation - try these commands:

Windows:

Command(s)

cd SOLR_INSTALL_DIRECTORY

bin\solr start -cloud -p 8983 -s "example\cloud\node1\solr"

bin\solr start -cloud -p 7574 -s "example\cloud\node2\solr" -z localhost:9983

*NIX:

Command(s)

cd SOLR_INSTALL_DIRECTORY

bin/solr start -cloud -p 8983 -s "example/cloud/node1/solr"

bin/solr start -cloud -p 7574 -s "example/cloud/node2/solr" -z localhost:9983

Tip: Documentation for this book can also be read online, see

https://synapticloop.github.io/panl/, or to download the latest copy of this book, see https://github.com/synapticloop/panl/releases/.

Copyright Notice

The data used in this book is based on 2mm mechanical pencil data hand curated by the author.  The data and associated Solr search results utilised in this book references manufacturers of writing instruments, specifically 2mm mechanical pencils.  The manufacturer's name, the name of the pencil (and/or model number) is a trademark and/or  is copyright of the manufacturer.

Additional datasets may have information that is copyright to the respective owners.

IMPORTANT: Whilst the project is released under the very liberal MIT licence, the sample mechanical data remains under the copyright of Synapticloop, and may not be used for commercial purposes.

~ ~ ~ * ~ ~ ~


TABLE OF CONTENTS

README.1ST        14

Welcome To Synapticloop Panl        14

Additional Panl Niceties        14

About This Book        16

Nomenclature Used Throughout This Book        16

Book Format Conventions        20

About Panl Server        22

How Many Facets Does Panl Support?        23

Panl URL Structure        24

In-Built Web Apps (Viewer / Explainer / Single Search Page)        26

About Panl Generator        33

About Apache Solr        33

Why Synapticloop Panl?        35

What is a LPSE (pronounced 'lapse') code?        36

The three types of LPSE codes        37

URL Path Nomenclature        37

Is Synapticloop Panl For Me?        38

What You Will Need        40

Download The Resources        40

The Panl Directory Structure        40

Panl Server Versions        43

Solr-Panl-9-x.x.x        43

Solr-Panl-8-x.x.x & Solr-Panl-7-x.x.x        43

Solr-Panl-6-x.x.x and earlier version        44

Quick Start - The 5 Steps        45

A Note On Running The Commands        46

Windows Commands        47

*NIX Commands        48

Next Steps        50

Getting Started        51

Downloading the Resources        51

The Solr server        52

The Panl server        52

Creating and Starting a Solr Cloud Instance        52

Creating a collection        55

Indexing the Data        56

Starting The Panl Server        57

When Something Goes Wrong        58

Checking the Logs        59

Deleting a collection        59

You will then need to re-create the collection, and re-index the data.        60

Restarting the Solr Panl tutorial process        60

Stopping Solr        60

Re-Starting Solr        61

Worst-case scenario        61

Next Steps        62

Additional Data        63

All Data Panl Server        63

Simple Date        64

Creating and Indexing the Data        64

panl.properties File Additions        65

Mechanical Pencils OR Facet        65

Creating and Indexing the Data        65

panl.properties File Additions        65

Mechanical Pencils More Facets        66

Creating and Indexing the Data        66

panl.properties File Additions        66

Book Store        67

A Walkthrough Example - The Book Store        68

0. High Level Requirements        69

1. Understanding the Dataset        70

2. Configure the Solr Index        72

3. Configure the Panl Server        77

The Default Search Page Configuration        78

The Generated panl.properties File        80

The Generated <panl_collection_url>.panl.properties File        80

4. Determine the Web Pages to Render        96

Author and Author Series        96

Author Listing        96

The Iterative Implementation Process        97

Working With Any Dataset        98

Supported Solr Data Types        98

Unsupported/Partially Supported Solr Field Types        99

Facet and Field Types        100

Fields        100

Regular Facets        101

BOOLEAN Facets        102

RANGE Facets        103

DATE Range Facets        104

OR Facets        105

Other Facet Options        106

A Brief Introduction To Solr (And Faceted Search)        108

Querying Data        108

The Solr Managed Schema        110

Determining the Appropriate FieldType and Attributes        115

The Solr Configuration File        119

Query Request Handler        120

Highlighting        121

Panl Configuration        123

The panl.properties file        123

The <panl_collection_url>.panl.properties file        124

The panl.properties Configuration File        126

Configuring The SolrJ Connector        127

Setting The Solr Server URL(s)        128

Enabling The Panl Results Testing URLs        129

The Panl Results Viewer        130

The Panl Results Explainer        131

Select one of the available CaFUPs links, which will then present the configuration for that particular CaFUP.        132

The Panl Single Page Search        132

Setting HTTP Status Message Verbosity        133

Binding Solr collections to Panl URL paths        134

The <panl_collection_url>.panl.properties Configuration File        136

Parameter and Operand Definitions        137

panl.param.query        137

panl.param.sort        137

panl.param.page        138

panl.param.numrows        139

panl.param.query.operand        140

panl.param.passthrough        140

panl.param.passthrough.canonical        141

panl.form.query.respondto        143

panl.include.single.facets        143

panl.include.same.number.facets        143

solr.default.query.operand        145

solr.facet.limit        145

solr.facet.min.count        146

solr.numrows.default        146

solr.highlight        146

Field Definitions        146

Properties Available for Fields        147

Facet Definitions        148

Properties Available for Facets        149

Regular Facet        151

Example        152

Decoding the URL        153

BOOLEAN Facet        153

Additional Properties        154

Example        155

Decoding the URL Without a Prefix or Suffix        155

Defining a Prefix and Suffix        156

Decoding the URL With a Prefix or Suffix        157

OR Facet        157

Additional Properties        157

The Difference Between Multi-valued Facets and OR Facets        158

Example        161

Decoding the URL        163

RANGE Facet        164

Additional Properties        165

Suppressing Individual Facet Values        167

Visualising the Properties        168

Decoding the URL - Range facet with an infix        170

Decoding the URL - Range facet without an infix        170

DATE Range Facet        170

Additional Properties        174

Example        174

Decoding the URL        175

Some Notes        176

About Value Replacements        176

On Field Validations        178

About Hierarchical Facets        178

About Sorting Facets        179

Putting It All Together (Technically)        181

A Brief Architecture        181

Production Readiness        182

Logging        182

Setting the Logging Levels        184

Generating SEO Friendly URLs        184

Data Dependencies        184

Passthrough URLs        184

Handling Errors        185

Panl LPSE URL Paths Explained        186

Parameter Query        187

Parameter Query Operand        188

Parameter Number of Rows        188

Parameter Page Number        190

Parameter Pass Through        191

Parameter Sort Order        192

Facets        193

Explaining A More Complex Example        195

Search Integration And The Panl Response Object        197

URL Bindings        197

Error Responses        197

Success Responses        198

The "active" JSON Object        199

Overall Integration and Implementation        201

The "active.facet" JSON Array (Regular facet object)        202

Integration and Implementation        203

The "active.facet" JSON Array (BOOLEAN facet object)        203

Integration and Implementation        204

The "active.facet" JSON Array (DATE Range facet object)        205

Integration and Implementation        206

The "active.facet" JSON Array (OR facet object)        207

Integration and Implementation        208

The "active.facet" JSON Array (RANGE facet object)        208

Integration and Implementation        209

The "active.numrows" JSON Object        210

Integration and Implementation        210

The "active.page" JSON Object        210

Integration and Implementation        211

The "active.query" JSON Object        211

Integration and Implementation        212

The "active.query" JSON Object        212

Integration and Implementation        213

The "active.sort" JSON Array        213

Integration and Implementation        213

The "available" JSON Object        214

The "available.date_range_facets" JSON Array        215

Integration and Implementation        216

The "available.facets" JSON Array        217

Integration and Implementation        218

The "available.range_facets" JSON Object        219

Integration and Implementation        219

The "pagination" JSON Object        226

Integration and Implementation        226

The "sorting" JSON Object        227

Integration and Implementation        229

Single Search Page Integration And The Panl Response Object        233

URL Bindings        233

Error Responses        234

Success Responses        234

The "lpse_lookup" JSON Object        235

The "lpse_order" JSON Array        236

Generating the LPSE URL path        236

More Facets Integration And The Panl Response Object        237

URL bindings        237

Building the Request URL        237

Determining The Facet Limit        238

Error Responses        240

Success Responses        240

The "facet" JSON Object        241

Implementation notes        241

Integrating An Existing Solr Schema        243

Using the Panl Generator        243

Running the Command Line Utility        243

Editing the Generated Files        247

Manually Creating the Configuration Files        247

Properties Quick Reference        249

Property References Format Explained        249

panl.bool.<lpse_code>.false        250

panl.bool.<lpse_code>.true        251

panl.collection.<solr_collection_name>        252

panl.date.<lpse_code>.days        253

panl.date.<lpse_code>.hours        254

panl.date.<lpse_code>.months        255

panl.date.<lpse_code>.next        256

panl.date.<lpse_code>.previous        257

panl.date.<lpse_code>.years        259

panl.decimal.point        260

panl.facet.<lpse_code>        260

panl.facetsort.<lpse_code>        261

panl.field.<lpse_code>        262

panl.form.query.respondto        263

panl.include.same.number.facets        263

panl.include.single.facets        264

panl.lpse.ignore        264

panl.lpse.length        265

panl.lpse.order        265

panl.multivalue.<lpse_code>        266

panl.name.<lpse_code>        266

panl.or.facet.<lpse_code>        267

panl.or.always.<lpse_code>        268

panl.param.numrows        268

panl.param.numrows.prefix        269

panl.param.numrows.suffix        269

panl.param.page        270

panl.param.page.prefix        270

panl.param.page.suffix        271

panl.param.passthrough        271

panl.param.passthrough.canonical        272

panl.param.query        272

panl.param.query.operand        273

panl.param.sort        273

panl.prefix.<lpse_code>        274

panl.range.facet.<lpse_code>        274

panl.range.infix.<lpse_code>        275

panl.range.max.<lpse_code>        276

panl.range.max.value.<lpse_code>        276

panl.range.max.wildcard.<lpse_code>        277

panl.range.min.<lpse_code>        278

panl.range.min.value.<lpse_code>        278

panl.range.min.wildcard.<lpse_code>        279

panl.range.prefix.<lpse_code>        280

panl.range.suffix.<lpse_code>        281

panl.range.suppress.<lpse_code>        281

panl.results.fields.<field_set>        282

panl.results.fields.default        282

panl.results.fields.empty        283

panl.results.testing.urls        283

panl.results.sort.fields        284

panl.status.404.verbose        284

panl.status.500.verbose        285

panl.suffix.<lpse_code>        285

panl.type.<lpse_code>        286

panl.when.<lpse_code>        286

solr.default.query.operand        287

solr.facet.limit        287

solr.facet.min.count        288

solr.highlight        288

solr.numrows.default        289

solr.search.server.url        289

solrj.client        289

Design Considerations        291

Collections And FieldSet URL Paths (CaFUPs)        292

In-Built Panl Results Testing URLs        293

Panl Server Startup        294

Error Messaging        295

404 - Not Found        295

500 - Internal Server Error        296

Testing vs. Production        298

Quick Middleware        298

Afterword        299

On The Tag-Line        299

On Documentation        299

Additional Functionality in the Pipeline        301

Appendices        305

Panl URL Bindings        305

User Defined URL Bindings        305

Panl Internal Functional URL Bindings        306

Internal Testing URL Bindings        307

Definitions        308

Command Line Options        313

Panl Server        313

Panl Generator        313

Usage Text        314

Sample .properties Files        317

The Sample panl.properties Files        317

The Sample <panl_collection_url>.panl.properties Files        318

Solr Version 8 & 7 Integration Notes        319

SolrJ        319

Solr Configuration files        319

JSON Response Object        319

Setting up a Solr 7 or 8 server        320

Additional Solr Version 7 Integration Notes        325

Solr Versions 6 and below        326

End Plate        328

~ ~ ~ * ~ ~ ~