User Tools

Site Tools


middleware:deploy:ed:ldap

Enterprise Directory LDAP Environment

Introduction

This document describes the process for installing the LDAP component of the Enterprise Directory project.
The LDAP environment consists of 7 separate machines:

  • 1 OpenLDAP instance that serves as the provider for replication.
  • 6 clustered consumer machines with an OpenLDAP ED-Unified instance and an Apache instance for LB monitoring.

Subversion

Installing the environment relies on having a working subversion client with access to the ED repository.
Instructions for installing a subversion client are available here.
If you need access to the ED repository you should contact Daniel Fisher.

Shell

Middleware uses BASH for its shell environment.
The first time you setup your environment svn will not be in your path

  1. Put the OpenSSL libraries in your library path:
    export LD_LIBRARY_PATH=/apps/local/openssl/lib
  2. Checkout the bashrc script:
    /apps/local/subversion/bin/svn checkout https://svn.middleware.vt.edu:8443/svn/ed/bash-conf $HOME/bash-conf
  3. Symlink a .bash_profile script from your home directory:
    ln -s $HOME/bash-conf/bash_profile $HOME/.bash_profile
  4. Symlink a .bashrc script from your home directory:
    ln -s $HOME/bash-conf/bashrc $HOME/.bashrc

All this script does is setup your PATH, PROMPT, and some useful aliases.
It can be easily adapted to other shells.
Note that if you are using this script and you change path locations in these instructions, then you may have to update this script.

Installing an ED-Unified slave instance

If something goes wrong during any of these steps, go here.

  1. Checkout the edunified project:
    svn co https://svn.middleware.vt.edu:8443/svn/ed/edunified /apps/src/edunified
  2. Change your current directory to the newly created edunified directory:
    cd /apps/src/edunified
  3. Change your current directory to the proper directory (trunk, branches, tags).
  4. Run the following command to build OpenLDAP and its dependencies:
    ./edunified.bash install slave /apps/local

    If you are building an instance on your personal machine, run the following:

    ./edunified.bash install slave PARENT_DIRECTORY_PATH

    PARENT_DIRECTORY_PATH is the fully qualified path of the parent directory you want your OpenLDAP instance to be installed under. Do not put trailing slashes on this path.

  5. You should get the following message: “ED-Unified install complete. Remember to set up the server certificates before you use this instance.” If you do not get this message, contact David Hawes.
  6. Set up certificates for the instance. See the custom builds section for information on setting up an instance other than ed, ed-master, ed-pprd, or ed-dev.
    1. cp ed-unified-cert.pem ed-unified-key.pem /apps/local/openldap-VERSION/etc/openldap
  7. Load the LDIF:
    ./edunified.bash load /apps/local LDIF_FILE
  8. You should get the following message: “ED-Unified load complete.”. If you do not get this message, contact David Hawes.
  9. Make sure that the authstats overlay is pointed to the proper ejb machine pool (dev, pprd, prod) in $OPENLDAP_HOME/etc/openldap/slapd.conf
  10. Start the instance:
    ed@ed-instance$ ./ed.bash start
  11. Ensure that each machine has client certificates set up properly. See slapd.conf for proper configuration.

Installing an ED-Unified master instance

If something goes wrong during any of these steps, go here.

  1. Checkout the edunified project:
    svn co https://svn.middleware.vt.edu:8443/svn/ed/edunified /apps/src/edunified
  2. Change your current directory to the newly created edunified directory:
    cd /apps/src/edunified
  3. Change your current directory to the proper directory (trunk, branches, tags).
  4. Run the following command to build OpenLDAP and its dependencies:
    ./edunified.bash install master /apps/local
  5. You should get the following message: “ED-Unified install complete. Remember to set up the server certificates before you use this instance.” If you do not get this message, contact David Hawes.
  6. Set up certificates for the instance. See the custom builds section for information on setting up an instance other than ed, ed-master, ed-pprd, or ed-dev.
    1. cp ed-master-cert.pem ed-master-key.pem /apps/local/openldap-VERSION/etc/openldap
  7. Load the LDIF:
    ./edunified.bash load /apps/local LDIF_FILE
  8. You should get the following message: “ED-Unified load complete.”. If you do not get this message, contact David Hawes.
  9. Start the instance:
    ed@ed-instance$ ./ed-master.bash start

Getting an ED-Unified LDIF Extract

Getting an Extract From an Existing ED-Unified Instance

  1. $OPENLDAP_HOME/sbin/slapcat -l ed-u.ldif

Custom Builds (not on ed-* machine)

  • These instructions are for building ED-Unified instances on machines that exist solely for running the Enterprise Directory. If you want to run an ED-Unified instance on a different machine, you may need to make a few changes to get it to run properly.
    1. In ed.bash, change SLAPD_USER to the user that will run OpenLDAP.
    2. Create a new case statement that corresponds to your hostname (by running the 'hostname' command) and has the correct IP addresses to listen on for your machine.
    3. For logs to work properly, you will need to add a similar line to your /etc/syslog.conf:
      # syslog.conf
      ...
      local4.*        -/apps/logs/edldap.log
      ...
    4. Create a self-signed certificate for the instance:
      openssl req -new -x509 -nodes -out ed-unified-cert.pem -keyout ed-unified-key.pem -days 365
    5. Copy the keypair:
      cp ed-unified-cert.pem ed-unified-key.pem $OPENLDAP_HOME/etc/openldap/
    6. Download the VT Middleware CA chain and copy it to the OpenLDAP instance:
      cp vt-cachain.pem $OPENLDAP_HOME/etc/openldap/

Installing the Apache Load Balancing Monitors

The Production Enterprise Directory requires each slave node to have an Apache instance that serves up monitor scripts. This allows the load balancer to determine if the ED-LDAP instances should be available to the production pool. These instructions detail the steps to install the Apache server.

  1. Checkout the edunified project:
    svn co https://svn.middleware.vt.edu:8443/svn/ed/edunified /apps/src/edunified
  2. Change your current directory to the newly created edunified directory:
    cd /apps/src/edunified
  3. Change your current directory to the proper directory (trunk, branches, tags).
  4. Change the PERL_VERSION variable in edunifed.bash to the version of Perl on the target machine. This can be determined with the following command:
    $ perl -v
  5. Run the following command to build Apache and its dependencies:
    ./edunified.bash monitor-install /apps/local

    If you get prompted by CPAN, simply accept the defaults.

    • For custom installs you must edit:
      • ${APACHE_HOME}/conf/httpd.conf and set the following variables: Listen IP address, user, and group
      • ${APACHE_HOME}/monitors/test-ed* and set the following variables: ED_IP
  6. Start the Apache instance by running:
    $ /apps/local/bin/apache.bash start
  7. To enable a particular type of ED-LDAP instance on the machine, copy the appropriate monitor script from the ${APACHE_HOME}/monitors directory to the $APACHE_HOME/cgi-bin directory. The following illustrates enabling ED-Auth on the machine:
    $ cp $APACHE_HOME/monitors/test-edauth $APACHE_HOME/cgi-bin
  8. To disable a particular type of ED-LDAP instance, simply remove the monitor script from the ${APACHE_HOME}/cgi-bin directory. The following illustrates disabling ED-Auth on the machine:
    $ rm $APACHE_HOME/cgi-bin/test-edauth

When Something Goes Wrong

  • Manually resetting an OpenLDAP instance.
    1. Delete all database files:
      rm $OPENLDAP_HOME/var/openldap-data/__db* \ 
         $OPENLDAP_HOME/var/openldap-data/*.bdb \
         $OPENLDAP_HOME/var/openldap-data/bdb-logs/log.*
    2. Fix the offending line in the LDIF file.
    3. Reload the instance.
  • On install, I do not get the message: “ED-Unified install complete. Remember to set up the server certificates before you use this instance.”
  • On loading, I do not get the message: “ED-Unified load complete.”

Other Platforms

Changes required to install on other platforms.

FreeBSD

  • Modify the edunified.bash script
    • set bash location:
      -#!/bin/bash
      +#!/usr/local/bin/bash
    • modify the openssl configure flag:
      -        ./config --prefix=${PREFIX} shared
      +        ./config --prefix=${PREFIX} no-fips shared
    • modify compile flags:
      -    CPPFLAGS="$CPPFLAGS -I${PREFIX}/include"
      -    LDFLAGS="$LDFLAGS -L${PREFIX}/lib"
      +    CPPFLAGS="$CPPFLAGS -I${PREFIX}/include -I/usr/local/include"
      +    LDFLAGS="$LDFLAGS -L${PREFIX}/lib -L/usr/local/lib"
  • After ED install, modify the ed.bash script
    • set bash location:
      -#!/bin/bash
      +#!/usr/local/bin/bash
  • Install the p5-perl-ldap module from packages
  • Create directory depend/perl
  • After apache monitor install, modify the apache.bash script
    • set bash location:
      -#!/bin/bash
      +#!/usr/local/bin/bash

Troubleshooting

The ED-LDAP instances rarely have issues, but if something goes wrong, it's easy to get them up and running again.

ED-LDAP environment

An ED-LDAP consumer (slave) has the following components:

  • an OpenLDAP instance
  • an Apache instance for monitoring

The ED-LDAP provider (master) has the following components:

  • an OpenLDAP instance
  • the syncprov overlay

isup.bash script

There is a script on each public ED-LDAP instance (not ed-master) that watches /apps/logs/edldap.log for changes and will send out text alerts and email if it detects that a log has not changed for five minutes. It will also attempt to restart the ED-LDAP instance. Note that if the server has not actually crashed, the server will not be restarted, and something else is probably wrong, such as a deadlock. This script exists to make sure the instances get restarted when the server crashes because of a bug or because it ran out of memory.

Note that when isup.bash detects that an instance hasn't written to the logs and sends an alert, it does not restart itself. You should login to the machine and do the following when the instance is back up:

$ sudo su - ed
$ isup.bash &

Restarting instances

If an instance needs to be restarted, simply login to the machine and:

$ sudo su - ed
$ ed.bash stop && ed.bash start

In the case that the instance is ed-master, you should:

$ ed-master.bash stop && ed-master.bash start

Removing an instance from the public pool

The Apache instance on each machine exists so the load balancer can perform health checks and remove machines that are down. To force an instance down, simply:

$ pkill isup.bash
$ apache.bash stop

When you are ready to put it back in the pool, run:

$ apache.bash start
$ isup.bash &

Replication

Replication is handled by syncrepl. In this configuration, consumers perform an LDAP Search request with an LDAP Sync control against the provider.

Determining if the provider and consumers are synced

If properly synced, the following will have equal contextCSNs.

ldapsearch -H ldap://ed-1.middleware.vt.edu:10389 -Z -b dc=vt,dc=edu -s base contextCSN 2>/dev/null | grep contextCSN: && \
ldapsearch -H ldap://ed-2.middleware.vt.edu:10389 -Z -b dc=vt,dc=edu -s base contextCSN 2>/dev/null | grep contextCSN: && \
ldapsearch -H ldap://ed-3.middleware.vt.edu:10389 -Z -b dc=vt,dc=edu -s base contextCSN 2>/dev/null | grep contextCSN: && \
ldapsearch -H ldap://ed-4.middleware.vt.edu:10389 -Z -b dc=vt,dc=edu -s base contextCSN 2>/dev/null | grep contextCSN: && \
ldapsearch -H ldap://ed-5.middleware.vt.edu:10389 -Z -b dc=vt,dc=edu -s base contextCSN 2>/dev/null | grep contextCSN: && \
ldapsearch -H ldap://ed-6.middleware.vt.edu:10389 -Z -b dc=vt,dc=edu -s base contextCSN 2>/dev/null | grep contextCSN: && \
ldapsearch -H ldap://ed-master.middleware.vt.edu:10389 -Z -b dc=vt,dc=edu -s base contextCSN 2>/dev/null | grep contextCSN:

Note: You will need to turn off certificate verification and use SASL EXTERNAL to bind as the LDAP manager.

Following Changes on the Provider

This will perform an LDAP Search request with an LDAP Sync control against the accesslog (which contains the changes on the master).

ldapsearch -H ldap://ed-master.middleware.vt.edu:10389 -Z -b cn=accesslog -E sync=rp

Note: You will need to use SASL EXTERNAL to bind as the LDAP manager.

Watching a specific entry for changes:

ldapsearch -H ldap://ed-master.middleware.vt.edu:10389 -Z -b uid=1152120,ou=people,dc=vt,dc=edu -E sync=rp

Note: You will need to use SASL EXTERNAL to bind as the LDAP manager.

Machine failures

If a machine fails for some reason such as a power outage, the instances should come up by themselves when the machines are rebooted, as the startup scripts are located in ”/apps/local/init.d”. If there are issues starting the OpenLDAP instances, ”/apps/logs/edldap.log” should state the reason.

BigBrother

We monitor all instances with BigBrother, and alerts will be sent out if any part of the instances go down.

middleware/deploy/ed/ldap.txt · Last modified: 2015/06/01 12:02 (external edit)