What rendering API do you use?
Well, if we suppose that your voxel corresponds to one cubic unit of your rendering API (i.e., each dimension of your voxel is 1 unit in length), you could define solid voxels by specifying an array of 3 int values (integers are used, since you have a 3D voxel grid composed of equally-sized cubes composed of 1 unit each, so no floating point values are required) that will correspond to the number of units to be traversed in each axis (X, Y and Z), for the voxel position to be reached. So, (0,0,0) would create a solid voxel at the origin, and (1,0,0) would create a solid voxel one unit along the +X axis, and 0 units along the Y and Z.
Incidentally, if you consider your voxel's "pivot point" (local-coordinate-space origin) and position it properly, you can make this triple-int value correspond to the offset vector from the world origin required for rendering the voxel. So the three integer values would give you the translation transformation required to position your voxel in World-space (which actually is all the information you need for rendering the voxel, since your voxel grid doesn't require rotation and scale transformations).
So, based on all this, your solid voxel can be defined as a 1x1x1 cube, that is transformed to World coordinates using the three integer values specified. You can read a file containing a list of three-int sets/arrays, each of which defines a solid voxel in world-space. Like this:
5 5 5
4 3 20
5 8 908
45 7 89
12 4 55
The rough code is:
Check If Blocks To are Solid
Yes - Don't Render
No - Render
Your non-solid voxels should not be checked-for, since they are non-renderable to begin with. You should just read the list of three-int arrays and store it in memory, then render a 1x1x1 cube at each of the specified locations. As for renderable faces, enable backface-culling in the rendering API so that the polygons that face away from the camera are not rendered. This is a hardware-accelerated process and is very fast and memory efficient. Just specify the cubes in world-space, and the backface-culling process will be responsible for clipping anything non-visible before the rendering begins.
Also, make sure you use the hardware acceleration offered by your API (OpenGL, DirectX etc). In DirectX, use a static vertex buffer and index buffer in order to specify your re-usable solid voxel, and apply the world-space transformation for each solid voxel in your scene inside an .fx file in order to optimize speed (since the buffer can be static and not require updates, while the .fx code can calculate the world-view-projection transformations for each of your solid voxels).
This shouldn't be too unsatisfactory in performance, especially if you use integral offsets and not involve rotation and scale transformations. Still, it largely depends on your voxel resolution, which is the amount of voxels specified in your voxel box/scene.
Also, make sure you pre-calculate the normals for your solid voxel cube (which should also be specified in triangles -- so it would be 12 triangles rather than 6 quads). Normals are required for the lighting calculations (among other things), so precalculating them requires less runtime calculations (although the difference is trivial for a cube consisting of 12 triangular faces, but still).
Hope this helps somewhat,
EDIT: Also, it's good to apply a frame-limiter so that you don't render more than 60 frames per second.