From 19228600f14eea433c54e17c164c4efe3a029d77 Mon Sep 17 00:00:00 2001
From: haoyuren <13851610112@163.com>
Date: Fri, 4 Jul 2025 03:17:39 -0700
Subject: Add GenderBench for group entropy equalization research

- Integrated GenderBench evaluation suite for gender bias testing
- Added modified MBPP.py for enhanced code evaluation
- Setup complete for implementing gender debiasing through entropy minimization
---
 genderbench/docs/source/probe_cards.rst | 63 +++++++++++++++++++++++++++++++++
 1 file changed, 63 insertions(+)
 create mode 100644 genderbench/docs/source/probe_cards.rst

(limited to 'genderbench/docs/source/probe_cards.rst')

diff --git a/genderbench/docs/source/probe_cards.rst b/genderbench/docs/source/probe_cards.rst
new file mode 100644
index 0000000..9c53a07
--- /dev/null
+++ b/genderbench/docs/source/probe_cards.rst
@@ -0,0 +1,63 @@
+.. _probe_cards:
+
+Probe Cards
+===========
+
+Each `Probe` is documented with its own ``README`` file. This document describes
+the schema used to create these files. We describe the documents field by field
+as they are written.
+
+- Abstract
+    Abstract succintly describes the main idea behind the probe.
+- Harms
+    Description of harms measured by the probe.
+- Use case
+    What is the use case for using LLMs in the context of the prompt.
+- Genders
+    What genders are considered.
+- Genders definition
+    How is the gender indicated in the texts (explicitly stated, gender-coded
+    pronouns, gender-coded names, etc).
+- Genders placement
+    Whose gender is being processed, e.g., author of a text, user, subject of
+    a text.
+- Language
+    Natural language used in the prompts / responses.
+- Output type
+    What is type of the output, e.g., structured responses, free text.
+- Modality
+    What is the modality of the conversation, e.g., single turn text
+    chats, tools, image generation.
+- Domain
+    What is domain of the data used, e.g., everyday life, healthcare, business.
+- Realistic format
+    Is the format of prompts realistic? Is it possible that similar requests
+    could be used by common users? Do the queries make practical sense outside
+    of the probing context?
+- Data source
+    How were the data created, e.g., human annotators, LLMs, scraping.
+- Size
+    Number of probe items.
+- Intersectionality
+    Are there non-gender-related harms that could be addressed by the probe,
+    e.g., race, occupation.
+- Folder
+    Where is the code located.
+- Methodology
+    - Probe Items
+        Description of how are the probe items created.
+    - Data
+        Description of the necessary data used to create the probe items.
+    - Evaluation
+        Description of the answer evaluation methodology.
+    - Metrics
+        Description of all the calculated metrics.
+- Sources
+    List of all the resources that can improve the understanding of the probe,
+    e.g., related papers or datasets.
+- Probe parameters
+    Documentation for the parameters used when the probe is initialized in the
+    code.
+- Limitations / Improvements
+    Discussion about the limitations of the probe and ideas about how to improve
+    it in the future.
-- 
cgit v1.2.3