Samba – Windows connects via IP address but not hostname

This one stumped me for a while, but it all came down to time.  I was authenticating Windows servers to Samba via Active Directory.  The NTP service was not running on the Samba service and its time was far out of sync with the Active Directory server.

Quickest fix is to update the time on the Samba server, sync the hardware clock, and then start NTP.

  • Shut down the NTP service if it’s running, then run
    • ntpdate -u <AD server IP address>
  • Sync hardware clock to system time
    • hwclock -w
  • Start NTP service
    • /etc/init.d/ntpd start

My problem was that NTPD wasn’t starting at boot so I changed that in chkconfig and all is well.

  • /sbin/chkconfig –levels 35 ntpd on

Active Directory is very sensitive to time fluctuations between servers.  Make sure all of your servers are keeping accurate time and you’ll minimize Samba and Active Directory issues.

Advertisements

Partitioning Zabbix tables in MySQL

My Zabbix server was in a world of hurt and Ricardo over at zabbixzone.com saved the day.  At the time, I was monitoring about 200 devices with a 4GB, 4CPU virtual machine.  This poor box was running everything, Zabbix server, Zabbix UI, and MySQL.  Needless to say, it was struggling. It needed some performance tuning.

I came across Ricardo’s post and immediately implemented a slightly altered version of the plan he laid out.  Check out his post, it’s fantastic.

The basic premise is that there are certain tables in Zabbix that collect a ton of data.  Not only do the tables get big, they slow down the UI due to long query times, etc.  If you utilize MySQL’s ability to partition these tables you not only make your queries quicker, you can easily keep your disk space under control too.  I’m sure there’s more benefits, but those 2 were enough to sell me on it.

MySQL has an optimization trick known as pruning where it will only look in the partitions that have the expected data in them.  This saves a lot of disk IO in the process.

When you have partitioned tables in MySQL you can drop old partitions and only the data from those dropped partitions is deleted.  In the case of the history tables in Zabbix, I have item history set to 7 days.  I have created weekly partitions and delete any partitions older than 2 weeks.

My environment monitors over 400 servers now, on the same modest VM, and my weekly history partitions have about 75m rows each.  Each week I drop a single partition and get back almost 5G of disk space.  Not only does this save disk space, but Zabbix no longer has to look through, potentially, 75m extra rows to get what it needs and the indexes on the tables are a lot smaller too.

Here’s a good example of the pruning by MySQL.

mysql> explain partitions select value from history_uint where itemid=18435 and clock<=1340923260 order by itemid,clock desc limit 4;
+----+-------------+--------------+----------------+-------+----------------+----------------+---------+------+-------+-------------+
| id | select_type | table        | partitions     | type  | possible_keys  | key            | key_len | ref  | rows  | Extra       |
+----+-------------+--------------+----------------+-------+----------------+----------------+---------+------+-------+-------------+
|  1 | SIMPLE      | history_uint | p201207wk01    | range | history_uint_1 | history_uint_1 | 12      | NULL | 21525 | Using where |
+----+-------------+--------------+----------------+-------+----------------+----------------+---------+------+-------+-------------+

This explain plan shows that MySQL only looked in 1 partition, only about 75m rows instead of a possible 225m+.  The index knocks that number down to about 21525. That’s a win as far as I’m concerned.

Now, I’m going to stop babbling and you should go check out Ricardo’s post at Zabbix Zone.


Zabbix – use web monitoring for FTP check

Zabbix web scenarios use cURL to check your web pages and cURL can be used to connect/download/upload to a FTP server, so I thought I could combine the two and see what happens. Well, this check works a lot better than the original FTP check I had created so I’d say it’s a success. As a bonus, you get download throughput and response time metrics for your FTP session checks.

Create web scenario

Here’s how you can set up web monitoring for FTP. First, create your web scenario under Configuration, Web. Select a host to assign this web check to and then select Create Scenario.
FTP create scenario

I kept everything at the default settings and created a single step to download a file from the FTP server as a test that everything is working properly.
FTP step 1

Application - web (or something similar)
Name - FTP check
Authentication - none (I pass username/password in URL)
Update interval - 60
Agent - Internet Explorer 6.0 ( default, haven't tried other agents)
Status - active
Variables - empty
Steps - 
     name - FTP check
     URL - ftp://username:password@servername.domain.com
     post - empty
     timeout - 15
     required - filename (this is the file you want to check for on the FTP server)
     status codes - 226 (FTP server response codes)

Trigger Configuration

Once your scenario is in place you can create a trigger to fire when something isn’t right. Select Configuration, Host and select the host you created the web scenario under. Select Triggers and Create Trigger. Name your new trigger and choose the expression to trigger off of. I usually choose the ‘failed step of scenario…’ expression and set it to ‘last value NOT N’ with a value of zero. This way anything that isn’t a success will cause a trigger to fire.

name - FTP check
expression - {HOST_A:web.test.fail[FTP check].last(0)}#0
event generation - normal
severity - high

FTP Trigger create

Once all of that is finished check your new monitor entry under Monitoring, Web. If you want to get a little more advanced with your FTP checks, you can add a trigger that fires when the response time is too long. Just a thought.

The web monitoring doesn’t have to be used only for monitoring actual web pages. cURL is very powerful and can allow for some great checks for your Zabbix environment. Learn more about cURL’s awesomeness here.

That’s it for now. Have fun with Zabbix web monitoring.


Zabbix 2.0 low level discovery – SNMP

Zabbix 2.0 makes it easy to add network devices, file systems, and SNMP devices for monitoring.  They call their new method low-level discovery and I will show you how I set it up so you can quickly start monitoring your environment.

I first showed you how to set up low-level discovery for file systems  and network interfaces.  Today, you’ll see how to set it up for SNMP.

Low level discovery of SNMP is a huge improvement over how you managed SNMP capable devices in previous Zabbix releases.  It makes monitoring SNMP devices so much easier and accounts for dynamic attributes with minimal modifications.

There are still some improvements to LLD for SNMP that need to be made but this new feature is seeing a lot of activity in the Zabbix forums so I’m hopeful that the Zabbix team will address them soon.

SNMP Discovery Rules

The easiest way to get started and to learn about low level discovery for SNMP devices is to grab a few templates from the Zabbix Appliance build.  I only recently discovered this virtual image of a fully functioning test zabbix server.  This pre-built Zabbix server can be quickly spun up on your hypervisor of choice for testing.

To make it even easier for you, here are the templates downloaded from the most recent appliance build.

Template_SNMP_Generic.xml
Template_SNMP_Interfaces.xml
Template_SNMP_Device.xml

Import these templates through your Zabbix UI under Configuration, Templates, Import and then link Template_SNMP_Generic and Template_SNMP_Interfaces to Template_SNMP_Device.  The idea is to add a single template (Template_SNMP_Device) to every host you want to have those items, triggers, graphs, etc.

LLD SNMP Template Screen

A time saving template trick I do is to create a template for each of my SNMP community strings and then link the Template_SNMP_Device to those community name templates.  That way you can use a {$SNMP_COMMUNITY} macro in each template to set the community string across many hosts easily.

The Zabbix Appliance SNMP templates use a combination of low level discovery and special OIDs to create a template that can provide some pretty good system and network information for most SNMP devices right out of the box.

For example, one template provides me with the following info for most SNMP devices this template is assigned to.  This information is provided for every interface discovered by the template rules.

  • Admin Status
  • Description
  • Inbound errors
  • incoming traffic
  • interface name
  • operational status
  • outbound errors
  • outgoing traffic

It’s nice to link one template to a network device and instantly have monitoring of dynamic interfaces almost immediately with no changes to the templates.

General SNMP Configuration

Using these example templates as guides I’ve created additional templates for other SNMP devices.  One thing you will likely encounter is the need to add MIBs to your Zabbix server for SNMP devices so that you can use human readable names as opposed to all numeric OIDs.  This is pretty straight forward.

First, check where your MIBs are located (most likely /usr/share/snmp/mibs, on CentOS/RHEL anyway):

net-snmp-config --default-mibdirs

Copy your MIB files to the location specified as your default mibdir

Add a reference to your new MIB files in /etc/snmp/snmp.conf in the format

mibs +NEW-SNMP-DEVICE

where NEW-SNMP-DEVICE comes from the first line of your MIB file.

NEW-MIB-DEVICE DEFINITIONS ::= BEGIN

Now you should be able to use a textual representation of your OID in your Zabbix items, etc.  The net-snmp page has a lot more information on this.

That’s it for now.  This should be enough info to make you dangerous…  I’ll add more as I do more with SNMP low level discovery.

Let me know if this helps you out or if you have any questions and if you know of a better file sharing site than filefactory, let me know. It’s kinda clunky and very slow. It was a quick decision and I would like something better for getting those files to you.