The Harvest Environment

Selective Harvest Protocol

Not every fruit on the tree is ready at the same time. The Selective Harvest Protocol equips robotic grippers with multi-modal sensing and a configurable harvest spec, ensuring that only fruit meeting your exact quality standard is picked on each pass—eliminating the waste and rework of traditional strip-harvesting.

Computer Vision on Robotic Grippers

• RGB + Depth Camera at 30 fps — Each end effector carries a synchronized stereo pair: a 12 MP RGB sensor for color analysis and a structured-light depth sensor for 3D pose estimation. The dual stream runs at 30 frames per second, providing enough temporal resolution to track fruit even as the arm swings through the canopy.
• On-arm Edge Inference — A compact NVIDIA Jetson Orin NX module mounted on the arm processes each frame in under 33 ms, running the detection, segmentation, and classification pipeline entirely on-device with no network round-trip.

Fruit Evaluation Pipeline

• Color Segmentation — Pixel-level HSV analysis calculates the percentage of red, blush, or variety-specific target color coverage across the visible surface of each fruit. For bicolor varieties like Honeycrisp, the model distinguishes between desirable red blush and sunburn discoloration.
• Size Estimation — Depth data triangulates fruit diameter in millimeters with +/- 2 mm accuracy. Under-size fruit (below the variety's pack-out threshold) is left on the tree for a later pick pass.
• Blemish Detection — A convolutional neural network trained on 120,000+ labeled images identifies hail damage, insect stings, sooty blotch, flyspeck, russet, and mechanical scuffing. Each blemish is classified by type and quantified as a percentage of surface area.
• Ripeness Index via NIR Spectroscopy — A miniaturized near-infrared spectrometer (900–1700 nm) mounted alongside the camera measures internal quality indicators: starch-iodine index, soluble solids (Brix), and firmness (via light scattering). These readings feed a composite ripeness index calibrated per variety.

Configurable Harvest Spec

Growers define exactly what constitutes a pickable fruit via a JSON instruction pushed to every robot in the fleet:

{
  "variety": "Honeycrisp",
  "min_red_pct": 60,
  "min_diameter_mm": 70,
  "max_blemish_pct": 5,
  "min_brix": 12.5,
  "max_firmness_lbf": 18,
  "min_firmness_lbf": 13,
  "starch_index_range": [2.0, 4.5]
}

The spec can be updated mid-harvest from the Fleet Command dashboard. Within 10 seconds, every gripper in the orchard recalibrates its pick/pass decision boundary.

Gentle Handling

• Pneumatic Soft-Grip Fingers — Silicone-membrane fingers inflated to precisely controlled pressure (0.3–0.8 bar depending on fruit firmness). The contact patch distributes force evenly, preventing point-load bruising.
• 6-Axis Articulated Arm — Full six-degree-of-freedom articulation allows the gripper to approach from below, beside, or above the fruit—whichever angle minimizes stem pull and contact with adjacent fruit or branches.
• <1% Bruise Rate — In field trials across 500,000+ picks, the pneumatic soft-grip system achieved a bruise rate of 0.7%—significantly below the 3–5% industry average for hand-picking.

Real-Time Quality Feedback Loop

• Continuous Quality Monitoring — Every picked fruit is photographed post-pick by a second camera inside the collection funnel. If the post-pick quality audit detects an uptick in bruising or stem damage, the system triggers an automatic adjustment cycle.
• AI Adjusts Gripper Pressure — The reinforcement learning controller reduces pneumatic pressure by 0.05 bar increments and observes the effect over the next 50 picks, converging on the optimal pressure for current conditions (fruit firmness, temperature, humidity).
• Approach Angle Optimization — If stem-pull defects increase, the motion planner shifts to a twist-and-lift detachment pattern rather than a direct pull, reducing stem damage by up to 40%.

H-2A Autopilot Agent

The H-2A temporary agricultural worker program is essential for most commercial orchards—and notoriously complex to administer. The H-2A Autopilot Agent uses AI to automate the entire lifecycle: from initial job order filing through payroll compliance, housing audits, and DOL reporting. What used to consume 200+ hours of administrative time per season now runs on autopilot.

AI Document Generation

• ETA Form 9142A & Appendix A — The agent pre-fills temporary labor certification applications using your orchard profile, historical acreage data, and crop-calendar dates. It validates every field against current DOL requirements before submission, flagging inconsistencies for human review.
• Job Orders — Automatically generates State Workforce Agency (SWA) job orders with correct SOC codes, wage rates, job descriptions, and working conditions. The AI ensures language matches DOL audit-safe templates while accurately reflecting your specific operation.
• Housing Attestation — Generates OSHA-compliant housing attestation documents, cross-referencing your housing inventory (beds, square footage, cooking facilities, sanitation) against the number of workers requested. Flags capacity shortfalls before you file.
• Transportation Plans — Calculates inbound/outbound transportation costs, generates reimbursement schedules, and produces compliant transportation plan documents that meet DOL's "most economical and reasonable" standard.

Compliance Monitoring

• Wage Rate Checks — Continuously monitors that all piece-rate and hourly earnings meet or exceed the Adverse Effect Wage Rate (AEWR), the prevailing wage, and the applicable state/federal minimum wage—whichever is highest. Alerts fire the instant any worker's effective hourly rate drops below the threshold.
• AEWR Update Tracking — The agent subscribes to DOL AEWR publications and automatically updates payroll configurations when new rates take effect, preventing the most common H-2A audit finding.
• Housing Inspection Scheduling — Tracks pre-occupancy and periodic inspection requirements, schedules inspections with certified inspectors, and stores signed inspection reports in the compliance vault for instant retrieval during audits.
• Three-Quarter Guarantee Tracking — Monitors each worker's cumulative hours against the three-quarter guarantee requirement, alerting managers when shortfalls are trending so they can reassign workers before the contract period ends.

Worker Matching

• Vetted Labor Contractor Network — The agent connects to Zooberry's network of licensed H-2A labor contractors and recruitment agencies. It matches your crop type, season window, and crew-size requirements against contractor availability and track record.
• Skill-Based Crew Composition — The matching algorithm considers worker experience (apple vs. cherry vs. pear), physical fitness assessments, and prior season performance scores to assemble crews optimized for your specific operation.
• Return Worker Prioritization — Workers who performed well in previous seasons are flagged for priority re-hire, reducing training time and improving Day 1 productivity.

Payroll Automation

• Piece-Rate Calculation — Integrates directly with Smart-Bin IoT weight sensors. Each picker's harvest weight is automatically converted to earnings at the agreed piece rate, with real-time running totals visible on the worker's mobile app.
• Hourly & Overtime Compliance — For hourly workers, the system enforces state-specific overtime rules (e.g., California's daily overtime after 8 hours, weekly after 40 hours) and applies the correct rate automatically.
• Piece-Rate Hourly Floor — When a piece-rate worker's effective hourly earnings fall below the AEWR, the system automatically tops up the difference and documents the adjustment for DOL compliance records.
• Multi-State Payroll — For operations spanning multiple states, the agent handles state tax withholding, workers' compensation insurance rate differences, and varying labor law requirements without manual configuration.