So there's this clever thing that mammals do. They breastfeed. And when milk enters the intestines it signals to the intestinal lining to loosen the links between cells (called tight junctions because cells are linked directly to each other rather than suspended in the extra-cellular matrix) so proteins and peptides can enter the bloodstream. Peptides like growth hormones and proteins like antibodies. So the mother can share her adaptive immune system (the immune system that remembers things like vaccinations) with her off-spring. It's only partial protection because the cells necessary to produce the antibodies are not shared, but it's better than nothing. A quick glance at the literature suggests that capric acid may be the causative agent in milk. I haven't spent much time looking though.
tl;dr if you have problems with your immune system try eliminating dairy, gluten, legumes, nightshades, coconut oil, palm oil. All of them at the same time.
Other agents that loosen up tight junctions are chitosan and gluten. Chitosan is a semi-synthetic polymer that has been investigated as a drug delivery vehicle. And you already know gluten. Wheat is by far the greatest source of gluten but celiacs also get sick from rye or barley. And they have too much carbohydrate and it's very easily digestible carbohydrate so you get the whole insulin-blood sugar roller coaster when you eat a fair amount of them (if you don't it can be an indicator of insulin insensitivity - you might want to get that checked out).
In addition lectins, a group of plant carbohydrate-binding proteins, bind to the surface of intestinal cells and may disrupt tight junctions or become transported into the blood in some other manner. Peanut lectin is famous for this.
Why am I even blabbing about all this? Well, if you have any kind of food allergies you might wonder how the hell these immune system provoking peptides or proteins end up inside your body. Supposedly things get broken down into monomer carbohydrates, individual fatty acids (from triglycerides), amino acids etc before being allowed into the body. Well, if your tight junctions are loose junctions then the doors are open for all allergens (bacteria still can't come through). So if I were you I would try giving up dairy, gluten grains, and various plant lectin containing yummies (I'm looking at you legumes and nightshades). That was it really. And sometimes paleo-people rant about "holes in your intestines! So very full of holes!" and now you know what the hell they are on about.
PS Yes, coconut oil is the darling of the paleo community and I heart it but if capric acid is how milk lets stuff in then coconut fat can do the same. And some animal fat. We can't all tolerate or be allergic to all the same things.