anl Server
Getting Started With Panl
A rather pleasing companion to the Apache® Solr® Faceted Search Engine
_____________
Author: Synapticloop - Version 2.0.0 (Release 1)
This book will rapidly get you up and running with a fully featured, SEO-friendly, keyword searchable, faceted search engine with an in-built, example search page to test it all out.
(Note: The most up-to-date version of this book can always be found here: https://github.com/synapticloop/panl/tree/main/src/book, and the online version here: https://synapticloop.github.io/panl/)
WHY?
Because...
/Caran d'Ache/true/Black/bDW/
looks so much nicer than...
q=*:*&facet.mincount=1&rows=10&facet.field=lead_size_indicator&facet.fie
ld=grip_material&facet.field=colours&facet.field=nib_shape&facet.field=d
iameter&facet.field=cap_shape&facet.field=brand&facet.field=mechanism_ty
pe&facet.field=length&facet.field=hardness_indicator&facet.field=grip_ty
pe&facet.field=cap_material&facet.field=lead_grade_indicator&facet.field
=tubing_material&facet.field=in_built_sharpener&facet.field=disassemble&
facet.field=category&facet.field=body_shape&facet.field=clip_material&fa
cet.field=mechanism_material&facet.field=lead_length&facet.field=body_ma
terial&facet.field=in_built_eraser&facet.field=grip_shape&facet.field=re
lative_weight&facet.field=name&facet.field=nib_material&facet.field=weig
ht&facet.field=variants&facet=true&fq=brand:"Caran+d'Ache"&fq=disassembl
e:"true"&fq=colours:"Black"&q.op=AND
TL;DR
If you are looking to get the quickest start on understanding how a Panl server is configured and up and running, then...
- GETTING UP TO SPEED
Start with the Quick Start - The 5 Steps section to get the Solr and Panl servers up and running and browse the Panl Results Viewer.
http://localhost:8181/panl-results-viewer/mechanical-pencils/default/ - UNDERSTAND
The Solr Fundamentals and Panl Fundamentals section introduces you to the concepts that lay the foundations for the rest of the book. Once you have a good understanding of the configuration and functionality, then there are two example walkthroughs:
- A FULL EXAMPLE
Work through the The Bookstore Walkthrough section to gain a deeper understanding of the complete process, from schema design, indexing, and configuration of both Panl and Solr. - PROGRAMMATIC INDEXING
Work through the Filesystem Indexing And Searching Walkthrough chapter to understand how to programmatically index files and commit them to Solr.
- INTEGRATE
Use your existing dataset to set up Panl with Integrating An Existing Solr Schema. - REFERENCE
Need help understanding the options? See the Panl Configuration part. - UPGRADE
See the Upgrading Your Panl Server Version section.
QUICK TIP
|
Tip: Documentation for this book can also be read online, see:
https://synapticloop.github.io/panl/
or, to download the copy of this book for the Panl version that you are using, see:
https://github.com/synapticloop/panl/releases/
Corrections and updates to this book, when done, will always be reflected online and the main GitHub branch - see:
|
~ ~ ~ * ~ ~ ~
Copyright Notices
MECHANICAL PENCIL DATA: The data used in this book is based on 2mm mechanical pencil data hand curated by the author. The data and associated Solr search results utilised in this book references manufacturers of writing instruments, specifically 2mm mechanical pencils. The manufacturer's name, the name of the pencil (and/or model number) is a trademark and/or is copyright of the manufacturer.
IMAGES, APPLICATIONS, AND SITE REFERENCES: Screenshots of applications and external websites are included in this book, they remain the copyright of the original owner and do not infer any association with, or endorsement of, Synapticloop, and/or vice versa.
Additional datasets may have information that is copyright to the respective owners.
IMPORTANT: Whilst the project is released under the very liberal MIT licence, the sample mechanical data remains under the copyright of Synapticloop, and may not be used for commercial purposes.
A NOTE ON URL ENCODINGS: Within this book if you are using the Firefox browser then the address bar will automatically decode any URL encoded space characters ('%20') - other browsers will keep the encoding in the address bar. Within this book, the majority of example URLs will not show the %20 encoding, however the associated href in the link will include these encodings.
~ ~ ~ * ~ ~ ~
TABLE OF CONTENTS
———————
PART 1:
INTRODUCTION
. 20
Nomenclature Used Throughout This Book 22
A Simple Faceted Search Example 31
A More Complex Faceted Search Example 32
Welcome To Synapticloop Panl 35
How Many Facets Does Panl Support? 39
In-Built Web Apps (Viewer / Explainer / Single Search Page) 43
What is a LPSE (pronounced 'lapse') code? 55
The three types of LPSE codes 56
Is Synapticloop Panl For Me? 57
—
PART 2:
GETTING UP AND RUNNING
. 59
The Panl Directory Structure 60
Solr-Panl-8-x.x.x & Solr-Panl-7-x.x.x 64
Solr-Panl-6-x.x.x and earlier version 64
A Note On Running The Commands 67
Latest Version Panl 2.0.0 and Solr 9.8.0 67
Previous Solr 9.*.* Versions Command Line Changes 67
Solr Versions Command Line Options 76
Creating and Starting a Solr Cloud Instance 77
Stopping and Restarting the Solr Panl tutorial process 84
Solr Configuration File Changes 89
Determining If A Solr Field Type Is Analysed 91
Overview of Indexed / Stored / Analysed Fields 93
The Relationship Between Configuration Files 96
Generating And Editing The Configuration Files 97
Keyword Search Field Configuration 99
Specific Field Keyword Search Configuration 105
<panl_collection_url>.panl.properties 106
Setting The Field To Allow A Specific Search 112
Summary of Panl Field Configuration Options 112
0. High Level Requirements 115
1. Understanding the Dataset 116
2. Configure the Solr Index 119
3. Configure the Panl Server 125
The Default Search Page Configuration 128
The Generated panl.properties File 130
The Generated <panl_collection_url>.panl.properties File 130
4. Determine the Web Pages to Render 156
The Iterative Implementation Process 157
Filesystem Indexing And Searching Walkthrough 159
1. Initialise a new project 160
2. Download and Configure the Solr Server 162
3. Download and Configure the Panl Server 166
4. Write the Indexing Code 169
6. Search and facet on the results 176
Creating and Indexing the Data 180
panl.properties File Additions 181
Mechanical Pencils OR Facet 181
Creating and Indexing the Data 182
panl.properties File Additions 182
Mechanical Pencils More Facets 182
Creating and Indexing the Data 183
panl.properties File Additions 183
Unsupported/Partially Supported Solr Field Types 188
A Note on Prefixes, Suffixes, and Infixes 189
Understanding Solr Configuration And Panl Integration 206
The Default Keyword Search 209
The Specific Solr Field Keyword Search 211
Determining the Appropriate FieldType and Attributes 219
Testing The Impact Of Indexed / Stored / Analysed 224
The Impact Of docValues (Schema Version 1.7+) 230
The Solr Configuration File 233
—
PART 4:
PANL CONFIGURATION IN DETAIL
. 238
The <panl_collection_url>.panl.properties file 240
The panl.properties Configuration File 242
Configuring The SolrJ Connector 243
Setting The Solr Server URL(s) 244
Enabling The Panl Results Testing URLs 245
The Panl Results Explainer 248
The Panl Single Page Search 248
Setting HTTP Status Message Verbosity 249
Binding Solr collections to Panl URL paths 250
The <panl_collection_url>.panl.properties Configuration File 252
Parameter and Operand Definitions 253
panl.param.passthrough.canonical 258
panl.include.single.facets 260
panl.include.same.number.facets 260
solr.default.query.operand 262
Properties Available for Fields 264
Properties Available for Facets 266
Decoding the URL With a Replacement Value 275
Defining a Prefix and Suffix 275
Decoding the URL With a Value Replacement and a Prefix or Suffix 276
The Difference Between Multi-valued Facets and OR Facets 277
Suppressing Individual Facet Values 288
Visualising the Properties 289
Decoding the URL - Range facet with an infix 291
Decoding the URL - Range facet without an infix 291
—
PART 5:
INTEGRATION AND PRODUCTION
. 304
Putting It All Together (Technically) 306
Setting the Logging Configuration and Levels 309
Upgrading Your Panl Server Version 310
Generating SEO Friendly URLs 310
Panl LPSE URL Paths Explained 312
Explaining A More Complex Example 321
Search Integration And The Panl Response Object 323
Generating The href Attribute 330
Generating The Anchor Text 330
"response.docs" JSON Object Results Integration 330
Example docs JSON Response 330
Example panl.fields JSON Response 332
Implementation and Integration 332
"panl.search" Keyword Search Integration 334
Implementation and Integration 336
"panl.search" Specific Solr Search Field Integration 336
Implementation and Integration 339
Integration and Implementation 342
"panl.active.facet" REGULAR facet object 343
Integration and Implementation 344
"panl.active.facet" BOOLEAN facet object 344
Integration and Implementation 345
"panl.active.facet" BOOLEAN Checkbox facet object 346
Example JSON - active facets 347
Integration and Implementation 348
"panl.active.facet" DATE Range facet object 349
Integration and Implementation 350
"panl.active.facet" OR facet object 351
Integration and Implementation 352
"panl.active.facet" RANGE facet object 352
Integration and Implementation 354
Integration and Implementation 355
Integration and Implementation 357
"panl.active.query" JSON Object 357
Integration and Implementation 358
"panl.active.query_operand" JSON Object 359
Integration and Implementation 360
"panl.active.sort" JSON Array 360
Integration and Implementation 362
"panl.available" JSON Object 363
"panl.available.date_range_facets" JSON Array 364
Integration and Implementation 365
"panl.available.facets" JSON Array 365
Example JSON - OR Separator 365
Integration and Implementation 366
"panl.available.range_facets" JSON Object 368
Integration and Implementation 368
"panl.pagination" JSON Object 375
Integration and Implementation 375
"panl.sorting" JSON Object 376
Integration and Implementation 378
Single Search Page Integration And The Panl Response Object 382
The "lpse_lookup" JSON Object 384
The "lpse_order" JSON Array 385
Generating the LPSE URL path 385
More Facets Integration And The Panl Response Object 386
Determining The Facet Limit 387
Lookahead Integration And The Panl Response Object 392
Integrating An Existing Solr Schema 396
Running the Command Line Utility 396
Editing the Generated Files 400
Manually Creating the Configuration Files 400
The <panl_collection_url>.panl.properties File 402
—
PART 6:
REFERENCE MATERIAL
. 404
Displaying a BOOLEAN facet as a checkbox 405
Minimum and Maximum values 407
Condensed Multivalued Paths 409
Example Properties - Original 409
Example Properties - Condensed 410
Example Properties - Original 413
Example Properties - Condensed 414
Analysed Facets (think Word Clouds) 415
Properties Quick Reference 417
Property References Format Explained 417
panl.bool.<lpse_code>.false 418
panl.bool.<lpse_code>.true 419
panl.bool.checkbox.<lpse_code> 420
panl.collection.<solr_collection_name> 421
panl.date.<lpse_code>.days 422
panl.date.<lpse_code>.hours 423
panl.date.<lpse_code>.months 424
panl.date.<lpse_code>.next 425
panl.date.<lpse_code>.previous 427
panl.date.<lpse_code>.years 428
panl.facetsort.<lpse_code> 431
panl.include.same.number.facets 434
panl.include.single.facets 434
panl.multivalue.<lpse_code> 437
panl.multivalue.separator. <lpse_code> 437
panl.or.always.<lpse_code> 439
panl.or.separator.<lpse_code> 440
panl.param.passthrough.canonical 445
panl.range.facet.<lpse_code> 448
panl.range.infix.<lpse_code> 449
panl.range.max.<lpse_code> 450
panl.range.max.value.<lpse_code> 451
panl.range.max.wildcard.<lpse_code> 451
panl.range.min.<lpse_code> 452
panl.range.min.value.<lpse_code> 453
panl.range.min.wildcard.<lpse_code> 454
panl.range.prefix.<lpse_code> 454
panl.range.suffix.<lpse_code> 455
panl.range.suppress.<lpse_code> 456
panl.results.fields.<field_set> 457
panl.results.fields.default 458
solr.default.query.operand 465
Decoding the Solr Query Parameters 471
f.<solr_field>.facet.mincount= 476
f.<solr_field_name>.facet.sort= 476
REGULAR (multivalued) Facets 477
Collections And FieldSet URL Paths (CaFUPs) 486
In-Built Panl Results Testing URLs 487
500 - Internal Server Error 490
Additional Functionality in the Pipeline 497
Panl Internal Functional URL Bindings 505
Internal Testing URL Bindings 507
The Sample panl.properties Files 518
The Sample <panl_collection_url>.panl.properties Files 519
Solr Version 8 & 7 Integration Notes 521
Setting up a Solr 7 or 8 server 522
Additional Solr Version 7 Integration Notes 527
~ ~ ~ * ~ ~ ~